Executors

Executors

This package contains the built-in executors for Chancy. Executors are responsible for managing the execution of jobs after they’ve been retrieved from a queue. A single worker can have multiple queues, each using its own executor, allowing you to run jobs in the way that best suits their needs.

Capabilities

Not all executors have the same capabilities, typically due to limits in how threads are implemented in Python. The following table summarizes the capabilities of each executor:

Executor Capabilities

Executor	Sync Jobs	Async Jobs	Cancellation [1]	Time Limits	Memory Limits
ProcessExecutor	✓	✓	Platform [3]	Automatic [3]	Platform [3]
AsyncExecutor	✗	✓	✓	Automatic	✗
ThreadedExecutor	✓	✓	✗	Cooperative [2]	✗
SubInterpreter	✓	✓	✗	Cooperative [2]	✗

[1]

Cancellation is always possible before a job is started. Cancellation here refers to the ability to stop a job that is actively running.

[2] (1,2)

Jobs must accept a chancy.job.QueuedJob context and call checkpoint() periodically.

[3] (1,2,3)

Process capabilities depend on operating-system signal and resource support. Use supports() to inspect them at runtime.

Capabilities can also be inspected programmatically:

from chancy.executors.base import Executor
from chancy.executors.thread import ThreadedExecutor

ThreadedExecutor.supports(
    Executor.Capability.COOPERATIVE_TIME_LIMITS
)

ProcessExecutor (Default)

chancy.executors.process.ProcessExecutor Optimized for CPU-bound jobs, uses separate processes for true parallelism.

AsyncExecutor

chancy.executors.asyncex.AsyncExecutor Optimized for IO-bound jobs like API calls and database operations. Uses asyncio for efficient concurrent execution. Thousands of jobs can be executed concurrently with limited resources.

ThreadedExecutor

chancy.executors.thread.ThreadedExecutor Suitable for IO-bound jobs that can’t use asyncio. Uses threads for concurrent execution.

SubInterpreterExecutor

chancy.executors.sub.SubInterpreterExecutor Experimental executor using Python sub-interpreters. Provides GIL avoidance with lower overhead than processes.

Custom Executors

You can implement your own executor by subclassing the chancy.executors.base.Executor class and implementing the push() method. Custom executors should declare their supported Capability values in capabilities. The shared execution preparation then rejects unsupported function types and resource limits consistently.

Submodules

• Executor
• ProcessExecutor
• AsyncExecutor
• ThreadedExecutor
• SubInterpreterExecutor