swh.alter.utils module#

swh.alter.utils.iter_swhids_grouped_by_type(swhids: Iterable[ExtendedSWHID], *, handlers: Mapping[ExtendedObjectType, Callable[[C], Iterable[T]]], chunker: Callable[[Collection[ExtendedSWHID]], Iterable[C]] | None = None) → Iterable[T][source]#

Work on a iterable of SWHIDs grouped by their type, running a different handler for each type.

The object types will be in the same order as in handlers.

Parameters:

  • swhids – an iterable over some SWHIDs

  • handlers – a dictionary mapping each object type to an handler, taking a collection of swhids and returning an iterable

  • chunker – an optional function to split the SWHIDs of same object type into multiple “chunks”. It can also transform the iterable into a more convenient collection.

Returns: an iterable over the handlers’ results

swh.alter.utils.filter_objects_missing_from_storage(storage: StorageInterface, swhids: Iterable[ExtendedSWHID]) → List[ExtendedSWHID][source]#
swh.alter.utils.get_filtered_objects(storage: StorageInterface, get_objects: Callable[[int], Collection[ExtendedSWHID]], max_results: int) → Collection[ExtendedSWHID][source]#

Call get_objects(limit) filtering out results with filter_objects_missing_from_storage.

If some objects were filtered, call the function again with an increasing limit until max_results objects are returned.