hymir.executor module

class hymir.executor.Executor

Bases: ABC

An executor is responsible for executing workflows and managing the state of the jobs in the workflow.

clear(workflow_id: str)

Erase all information associated with the given workflow_id, only if the workflow has finished.

static job_state(workflow_id: str, job_id: str) → JobState

Get the state of a specific job within a workflow.

Parameters:

  • workflow_id – The unique identifier for the workflow.

  • job_id – The unique identifier for the job.

classmethod job_states(workflow_id: str, job_ids: list[str] = None) → dict[str, JobState]

Fetch multiple JobStates at once, returning a mapping of {job_id: job_state}.

If no job_ids are provided, all jobs in the workflow will be returned.

This function should be used whenever more than a single job needs to be checked, as it’s typically much more efficient.

outputs(workflow_id: str) → dict[str, list[Any]]

Get all outputs from all jobs that ran as part of the given workflow.

If outputs are missing - for example because the job that provided them failed - they will be ignored.

Outputs are always lists of values.

Parameters:

workflow_id – The unique identifier for the workflow.

progress(workflow_id: str) → tuple[int, int]

Gets the progress of a workflow, returning the number of jobs that have finished and the total number of jobs. It does not matter if a job failed or succeeded, only that it has finished executing.

Parameters:

workflow_id – The unique identifier for the workflow.

abstract run(workflow: Workflow) → str

Run a workflow and return a unique identifier which can be used to track the progress of the workflow.

Parameters:

workflow – The workflow to run.

Returns:

The unique identifier for the workflow.

static store_job_state(workflow_id: str, job_id: str, state: JobState)

Store the state of a specific job within a specific run of a workflow.

Parameters:

  • workflow_id – The unique identifier for the workflow.

  • job_id – The unique identifier for the job.

  • state – The JobState to store.

static store_workflow(workflow_id: str, workflow: Workflow)

Store the workflow for the executor to retrieve later.

Parameters:

  • workflow_id – The unique identifier for the workflow.

  • workflow – The workflow to be stored.

static store_workflow_state(workflow_id: str, state: WorkflowState)

Store the state of a workflow.

Parameters:

  • workflow_id – The unique identifier for the workflow.

  • state – The WorkflowState to store.

wait(workflow_id: str, *, block: bool = True, sleep: int = 5)

Wait until the given workflow has completed.

By default, this function blocks until the workflow has completed, sleeping sleep seconds between checks.

If block is False, the function will return immediately after checking the state of the workflow.

Parameters:

  • workflow_id – The unique identifier for the workflow.

  • block – Should the call block or not.

  • sleep – The number of seconds to sleep for between checks.

Returns:

static workflow(workflow_id: str) → Workflow

Retrieve the workflow from the executor.

Parameters:

workflow_id – The unique identifier for the workflow.

Returns:

The workflow.

static workflow_state(workflow_id: str) → WorkflowState

Get the state of a workflow.

Parameters:

workflow_id – The unique identifier for the workflow.

class hymir.executor.JobState(status: ~hymir.executor.JobState.Status = Status.PENDING, retries: int = 0, context: dict[str, ~typing.Any] = <factory>, exception: str | None = None)

Bases: object

Holds the state for a specific job, within a specific run of a workflow.

class Status(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: IntEnum

The status of a job in a workflow.

FAILURE = 3
PENDING = 0
STARTING = 1
SUCCESS = 2
context: dict[str, Any]

Arbitrary data that can be stored with the job for subsequent runs.

classmethod deserialize(data: str) → JobState

Deserialize a JobState from a JSON string.

exception: str | None = None

Exception information if the job failed.

property is_finished: bool

Check if the job has finished executing.

retries: int = 0

The total number of times this job has been retried.

serialize() → str

Serialize the JobState to a JSON string.

status: Status = 0

The last known status of the job.

class hymir.executor.WorkflowState(status: Status = Status.PENDING)

Bases: object

Holds the state for a specific run of a workflow.

class Status(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: IntEnum

The status of a workflow.

FAILURE = 4
PENDING = 0
RUNNING = 1
SUCCESS = 3
classmethod deserialize(data: str) → WorkflowState

Deserialize a WorkflowState from a JSON string.

property is_finished: bool

Check if the workflow has finished executing.

serialize() → str

Serialize the WorkflowState to a JSON string.

status: Status = 0

The last known status of the workflow.