Pruner

class chancy.plugins.pruner.Pruner(rule=None, *, maximum_to_prune=10000, poll_interval=60)

Bases: Plugin

A plugin that prunes stale data from the database.

Note

This plugin is enabled by default, you only need to provide it in the list of plugins to customize its arguments or if no_default_plugins is set to True.

from chancy import Chancy
from chancy.plugins.leadership import Leadership
from chancy.plugins.pruner import Pruner

app = Chancy("postgresql://localhost/postgres", plugins=[
    Leadership(),
    Pruner(
        (Pruner.Rules.Queue() == "default") & (Pruner.Rules.Age() > 60)
    )
])

The pruner only prunes jobs that have finished, either succeeding or failing. Jobs that are pending, running, or waiting to be retried are never pruned. When the pruner runs, it will also call the chancy.plugin.Plugin.cleanup() method on any plugins that implement it, allowing them to clean up any data that is no longer needed such as completed workflows.

Rules

You can use simple rules, or combine them using the | and & operators to create complex rules. Parenthesize each comparison when combining rules. Age is measured from the job’s creation time, not its completion time.

For example, to prune jobs that are older than 60 seconds:

age_pruner = Pruner(Pruner.Rules.Age() > 60)

Or to prune jobs that are older than 60 seconds and are in the “default” queue:

queue_pruner = Pruner(
    (Pruner.Rules.Queue() == "default") & (Pruner.Rules.Age() > 60)
)

Or to prune jobs that are older than 60 seconds and are in the “default” queue, or any finished myapp.tasks.update_cache job regardless of age:

job_pruner = Pruner(
    ((Pruner.Rules.Queue() == "default") & (Pruner.Rules.Age() > 60)) |
    (Pruner.Rules.Job() == "myapp.tasks.update_cache")
)

By default, the pruner will run every 60 seconds and will remove up to 10,000 finished jobs in a single run whose creation time is more than 24 hours (86,400 seconds) ago. A job that takes longer than 24 hours to finish is eligible on the next pruning pass after completion; there is no additional retention period measured from completion.

Tip

By default, only an Age rule will be covered by an index. If you use multiple rules, you may need to create additional indexes to improve performance on busy queues.

param rule:

The rule that the pruner will use to match jobs.

param maximum_to_prune:

The maximum number of jobs to prune in a single run of the pruner.

param poll_interval:

The interval in seconds between each run of the pruner.

Rules

alias of JobRules

static get_dependencies()

Get the identifiers of all plugins this plugin depends on, if any.

Plugins that depend on the presence of other plugins will refuse to start if those dependencies are not met.

Return type:

list[str]

static get_identifier()

Returns a unique identifier for this plugin.

This identifier should be unique across all active plugins. If a custom plugin provides compatible functionality to a built-in plugin, it may use the same identifier as the built-in plugin.

Return type:

str

async prune(chancy, cursor)

Prune stale records from the database.

Parameters:

• chancy (Chancy) – The Chancy application.

• cursor (AsyncCursor[dict[str, Any]]) – The database cursor to use for the operation.

Returns:

The number of rows removed from the database

Return type:

int

async run(worker, chancy)

Runs the plugin.

This function can and should run indefinitely, as it will be cancelled when the worker is stopped.

Parameters:

• worker (Worker)

• chancy (Chancy)

Parameters:

• rule (SQLAble | None)

• maximum_to_prune (int)

• poll_interval (int)