swh.scheduler.simulator package

Submodules

• swh.scheduler.simulator.common module
  • SimulationReport
    • SimulationReport.DURATION_THRESHOLD
    • SimulationReport.total_visits
    • SimulationReport.visit_runtimes
    • SimulationReport.scheduler_metrics
    • SimulationReport.visit_metrics
    • SimulationReport.latest_snapshots
    • SimulationReport.record_visit()
    • SimulationReport.record_metrics()
    • SimulationReport.uneventful_visits
    • SimulationReport.runtime_histogram()
    • SimulationReport.metrics_plot()
    • SimulationReport.metrics_csv()
    • SimulationReport.format()
  • Task
    • Task.visit_type
    • Task.origin
    • Task.backend_id
  • TaskEvent
    • TaskEvent.task
    • TaskEvent.status
    • TaskEvent.eventful
  • Queue
    • Queue.slots_remaining()
  • Environment
    • Environment.report
    • Environment.scheduler
    • Environment.time
• swh.scheduler.simulator.origin_scheduler module
  • scheduler_runner_process()
  • scheduler_journal_client_process()
• swh.scheduler.simulator.origins module
  • generate_listed_origin()
  • OriginModel
    • OriginModel.MIN_RUN_TIME
    • OriginModel.MAX_RUN_TIME
    • OriginModel.PER_COMMIT_RUN_TIME
    • OriginModel.EPOCH
    • OriginModel.seconds_between_commits()
    • OriginModel.get_last_update()
    • OriginModel.get_current_snapshot_id()
    • OriginModel.load_task_characteristics()
  • lister_process()
  • load_task_process()
• swh.scheduler.simulator.task_scheduler module
  • scheduler_runner_process()
  • scheduler_listener_process()

Module contents

This package runs the scheduler in a simulated environment, to evaluate various metrics. See Software Heritage Scheduler Simulator.

This module orchestrates of the simulator by initializing processes and connecting them together; these processes are defined in modules in the package and simulate/call specific components.

swh.scheduler.simulator.update_metrics_process(env: Environment, update_interval: int) → Generator[Event, None, None]

Update the scheduler metrics every update_interval (simulated) seconds, and add them to the SimulationReport

swh.scheduler.simulator.worker_process(env: Environment, name: str, task_queue: Queue, status_queue: Queue) → Generator[Event, Task, None]

A worker which consumes tasks from the input task_queue. Tasks themselves send OriginVisitStatus objects to the status_queue.

swh.scheduler.simulator.setup(env: Environment, scheduler_type: str, policy: str | None, workers_per_type: Dict[str, int], task_queue_capacity: int, min_batch_size: int, metrics_update_interval: int)

swh.scheduler.simulator.fill_test_data(scheduler: SchedulerInterface, num_origins: int = 100000)

Fills the database with mock data to test the simulator.

swh.scheduler.simulator.run(scheduler: SchedulerInterface, scheduler_type: str, policy: str | None, runtime: int | None)