Skip to content

Scheduler Overview

┌─────────────────────────────┐
│ Scheduler Process │
│ - Checks schedules │
│ - Calculates next run │
│ - Enqueues via broker │
└──────────────┬──────────────┘
│ enqueue_async()
▼
┌─────────────────────────────┐
│ PostgreSQL │
│ - tasks table │
│ - schedule_state table │
└──────────────┬──────────────┘
│ NOTIFY
▼
┌─────────────────────────────┐
│ Workers │
│ - Execute scheduled tasks │
└─────────────────────────────┘

The scheduler:

  1. Runs as a separate process/dyno from workers
  2. Checks configured schedules at regular intervals
  3. Enqueues tasks when schedules are due
  4. Tracks state in database to prevent duplicates

Add a ScheduleConfig to your AppConfig:

from horsies.core.models.schedule import ScheduleConfig, TaskSchedule, DailySchedule
from datetime import time
config = AppConfig(
broker=PostgresConfig(...),
schedule=ScheduleConfig(
enabled=True,
check_interval_seconds=1,
schedules=[
TaskSchedule(
name="daily-cleanup",
task_name="cleanup_old_data",
pattern=DailySchedule(time=time(3, 0, 0)),
timezone="UTC",
),
],
),
)
Terminal window
horsies scheduler myapp.instance:app --loglevel=INFO

Run separately from workers. One scheduler per cluster is sufficient.

Each schedule has a unique name used for state tracking:

TaskSchedule(
name="daily-report", # Must be unique
task_name="generate_report",
...
)

The task_name must match a registered @app.task():

@app.task("generate_report")
def generate_report() -> TaskResult[str, TaskError]:
...
TaskSchedule(
name="...",
task_name="generate_report", # Must match decorator
...
)

How often the scheduler checks for due schedules:

ScheduleConfig(
check_interval_seconds=1, # Check every second
...
)

Range: 1-60 seconds. Lower values provide better precision but more database queries.

The schedule_state table tracks:

Field Purpose
schedule_name Schedule identifier
last_run_at When schedule last executed
next_run_at When schedule should run next
last_task_id ID of most recent enqueued task
run_count Total execution count
config_hash Detects configuration changes

This prevents duplicate executions when:

  • Scheduler restarts
  • Multiple schedulers run (advisory locks serialize)
  • Network issues cause delays

Each slot has a deterministic task id, and the row stores a fingerprint of the enqueue. A scheduler that stops after the enqueue of a slot and before it saves the state fires that slot again at its next start. With the same command, the enqueue returns the existing task. With a different command, the broker returns PAYLOAD_MISMATCH. The command is different when the next start runs another horsies version, or when the schedule’s kwargs or the task’s options changed.

The slot already has its task, so the scheduler counts the slot as fired:

  1. It logs one warning that names the schedule, the slot and the task id.
  2. It records the run with that task id and moves to the next slot.
  3. The existing task keeps the command that it was enqueued with.

State rows are created at scheduler startup. If a schedule’s row creation fails there (for example a transient database error during start()), the schedule is logged as failed and the tick loop recreates the missing row on a later tick — each tick checks for enabled schedules without a state row and re-runs their initialization. The same check recovers a row deleted externally. The recreated row’s next_run_at is computed from the current time, so no missed-slot backlog is replayed.

When catch_up_missed=True, missed runs are executed:

TaskSchedule(
name="hourly-sync",
task_name="sync_data",
pattern=IntervalSchedule(hours=1),
catch_up_missed=True, # Execute missed runs
)

If the scheduler was down for 3 hours, it will enqueue 3 tasks on restart (bounded by max_catch_up_runs).

When catch_up_missed=False (default), missed runs are dropped: after downtime the scheduler fires at most one run — the most recent due slot — logs how many older slots were skipped, and resumes at the next future slot. Slot alignment is preserved, so task ids stay deterministic.

Each schedule can have its own timezone:

TaskSchedule(
name="morning-report",
task_name="send_report",
pattern=DailySchedule(time=time(9, 0, 0)),
timezone="America/New_York", # 9 AM Eastern
)

Uses Python’s zoneinfo module. Default is “UTC”.

Schedules are validated at startup:

  • Task must be registered
  • Queue must be valid (CUSTOM mode)
  • Required arguments must be provided
  • Kwargs must encode against the task’s types and fit payload.reject_bytes
# This will fail at scheduler start:
TaskSchedule(
name="bad",
task_name="nonexistent_task", # Not registered
...
)

The scheduler encodes the kwargs of each schedule once at startup and sends that text for every slot until the next start. A kwargs model field with a default_factory, or a validator that reads the clock, runs once per start, not once per fire. If such a validator refuses the value later, the task is enqueued and fails at the worker with WORKER_SERIALIZATION_ERROR.

A scheduled run stores the options of its task, as a plain .send() of the task does. The worker reads timeout_ms, retry_policy, execution and cancel_grace_ms from the row, so a scheduled run gets the same timeout, retries and execution class as a sent one. A schedule has no options of its own: set them on @app.task().

@app.task() encodes the options once, when the task is defined. The scheduler takes that text at startup and sends it for every slot until the next start. If the options do not encode, @app.task() raises ValueError with the task name and the cause, and the task is not registered. horsies scheduler then exits 1 at startup: the startup check reports the module import with MODULE_EXEC_ERROR, and Scheduler.start() refuses the schedule with TASK_NOT_REGISTERED.

The scheduler handles SIGTERM/SIGINT for clean shutdown:

Terminal window
# Sends SIGTERM
kill <scheduler_pid>

Current schedule check completes, then scheduler exits.