Executor

class chancy.executors.base.ConcurrentExecutor(worker, queue)[source]

Bases: Executor, ABC

Base class for executors based off of the concurrent.futures.Executor class with common functionality.

Parameters:
async cancel(ref)[source]

Attempt to cancel every running execution of the referenced job.

It’s not guaranteed that the job will be cancelled, as it may have already completed by the time this method is called or the executor may not support cancellation.

Parameters:

ref (Reference) – The reference to the job to cancel.

async cancel_execution(job)[source]

Attempt to cancel only the execution identified by job.claim_id.

Recovery uses this optional hook instead of job-wide cancellation. Custom executors must match both the job ID and claim ID. The default does nothing; database fencing still rejects the abandoned execution’s updates when cancellation is unsupported or unsuccessful.

Parameters:

job (QueuedJob)

get_running_jobs()[source]

Get all jobs currently running in this executor.

Returns:

A list of running jobs.

Return type:

list[QueuedJob]

classmethod job_wrapper(job)[source]

Run a job with cooperative limit handling.

Parameters:

job (QueuedJob)

Return type:

tuple[QueuedJob, Any]

jobs: dict[Future, QueuedJob][source]
async stop()[source]

Stop the executor, giving it a chance to clean up any resources it may have allocated to running jobs.

It is not safe to use the executor after this method has been called.

class chancy.executors.base.Executor(worker, queue)[source]

Bases: ABC

The base class for all executors.

Executors are responsible for managing the execution of jobs after they’ve been retrieved from a queue.

See ProcessExecutor and AsyncExecutor for examples of built-in executors.

Parameters:
class Capability(*values)[source]

Bases: IntFlag

Features supported by an executor.

ASYNC_JOBS = 2[source]
AUTOMATIC_TIME_LIMITS = 8[source]
CANCELLATION = 4[source]
COOPERATIVE_TIME_LIMITS = 16[source]
MEMORY_LIMITS = 32[source]
NONE = 0[source]
SYNC_JOBS = 1[source]
abstractmethod async cancel(ref)[source]

Attempt to cancel every running execution of the referenced job.

It’s not guaranteed that the job will be cancelled, as it may have already completed by the time this method is called or the executor may not support cancellation.

Parameters:

ref (Reference) – The reference to the job to cancel.

async cancel_execution(job)[source]

Attempt to cancel only the execution identified by job.claim_id.

Recovery uses this optional hook instead of job-wide cancellation. Custom executors must match both the job ID and claim ID. The default does nothing; database fencing still rejects the abandoned execution’s updates when cancellation is unsupported or unsuccessful.

Parameters:

job (QueuedJob)

capabilities = 0[source]
property concurrency: int[source]
property free_slots: int[source]
classmethod get_capabilities()[source]

Return the features supported by this executor.

Return type:

Capability

abstractmethod get_default_concurrency()[source]

Get the default concurrency level for this executor.

This method is called when the queue’s concurrency level is set to None. It should return the number of jobs that can be processed concurrently by this executor.

Return type:

int

classmethod get_function_and_kwargs(job)[source]

Find the function which should be executed for the given job and return its keyword arguments.

Subclasses can override this method to resolve the function differently or to inject additional keyword arguments, such as application-level dependencies, before the job runs. Every built-in executor goes through this method via prepare_job_for_execution().

Note

This method runs wherever the executor runs the job: in the worker’s event loop, in a pool thread, in a sub-interpreter or in a child process. For process and sub-interpreter executors, the executor subclass must be importable by name, and injected resources should be created in that execution context (for example from on_initialize_worker). Async and threaded executors share the worker’s memory; injected resources must be compatible with the event loop or thread where the job runs.

Parameters:

job (QueuedJob) – The job instance to get the function and arguments for.

Returns:

A tuple containing the function and its keyword arguments.

Return type:

tuple[Callable, dict]

static get_limit(job, limit_type)[source]

Return the first configured limit of a given type.

Parameters:
Return type:

int | None

get_running_job(ref)[source]

Get a running job by its reference.

Parameters:

ref (Reference) – The reference to the job to find.

Returns:

The job if found, None otherwise.

Return type:

QueuedJob | None

abstractmethod get_running_jobs()[source]

Get all jobs currently running in this executor.

Returns:

A list of running jobs.

Return type:

list[QueuedJob]

is_job_running(ref)[source]

Check if a job is currently running in this executor.

Parameters:

ref (Reference) – The reference to the job to check.

Returns:

True if the job is running, False otherwise.

Return type:

bool

async on_job_completed(*, job, exc=None, result=None)[source]

Called when a job has completed.

This method should be called by the executor when a job has completed execution. It will update the job’s state in the queue and handle retries if necessary.

Parameters:

  • job (QueuedJob) – The job that has completed.

  • exc (BaseException | None) – The exception that was raised during execution, if any.

  • result (Any) – The result of the job, if any.

async on_job_starting(job)[source]

Called when a job has been retrieved from the queue and is about to start.

Parameters:

job (QueuedJob)

Return type:

QueuedJob

classmethod prepare_job_for_execution(job)[source]

Validate and prepare a job according to executor capabilities.

Parameters:

job (QueuedJob)

Return type:

tuple[QueuedJob, Callable, dict]

abstractmethod async push(job)[source]

Push a job onto the job pool.

Parameters:

job (QueuedJob)

static run_function(func, kwargs)[source]

Run a synchronous or asynchronous function to completion.

Parameters:
Return type:

Any

abstractmethod async stop()[source]

Stop the executor, giving it a chance to clean up any resources it may have allocated to running jobs.

It is not safe to use the executor after this method has been called.

classmethod supports(capability)[source]

Return whether this executor supports all given capabilities.

Parameters:

capability (Capability)

Return type:

bool