Skip to content

Configuring Horsies

  • PostgreSQL 14+
  • Python 3.13+
  • libpq 17+ for a time limit on the cancel request of an interrupted query. The horsies[pg-binary] extra bundles its own libpq. See psycopg and libpq
Terminal window
uv add horsies
from pydantic import SecretStr
from horsies import Horsies, AppConfig, IsolatedChildren, PostgresConfig, WorkerProfile
config = AppConfig(
broker=PostgresConfig(
database_url=SecretStr("postgresql+psycopg://user:password@localhost:5432/mydb"),
),
worker_profiles={
"default": WorkerProfile(isolated=IsolatedChildren(count=8)),
},
)
app = Horsies(config)

worker_profiles names the settings that a worker starts with: here, one profile with 8 children. See Worker Profiles.

If your provider gives you a transaction-pooled PgBouncer URL, keep ordinary SQL traffic on that pooled URL and give Horsies a second direct/session-capable URL for schema setup and LISTEN/NOTIFY.

from horsies import Horsies, AppConfig, PostgresConfig
config = AppConfig(
broker=PostgresConfig(
database_url=SecretStr(os.environ["DATABASE_URL_POOLED"]),
session_database_url=SecretStr(os.environ["DATABASE_URL_DIRECT"]),
pgbouncer_transaction_mode=True,
),
)
app = Horsies(config)

Managed providers, such as PlanetScale Postgres, may show separate pooled and direct connection strings. Verify the current ports and URLs in your provider dashboard or docs. Workers do not support a PgBouncer-only URL because PostgreSQL LISTEN requires a persistent session.

Different operations have different urgency levels can be defined with priority values. 1-100 where 1 is priority numero uno.

from horsies import Horsies, AppConfig, PostgresConfig, QueueMode, CustomQueueConfig
config = AppConfig(
queue_mode=QueueMode.CUSTOM,
custom_queues=[
CustomQueueConfig(name="urgent", priority=1, max_concurrency=10),
CustomQueueConfig(name="standard", priority=50, max_concurrency=20),
CustomQueueConfig(name="low", priority=100, max_concurrency=5),
],
broker=PostgresConfig(
database_url=SecretStr("postgresql+psycopg://user:password@localhost:5432/db_name"),
),
)
app = Horsies(config) # use this app instance for decorating task
Queue Priority Use Case
urgent 1 The most important queue
standard 50 Things in between
low 100 The least important, can wait
app.discover_tasks([
"myapp.tasks",
"myapp.workflows",
])

discover_tasks records module paths for later import by the worker. Dotted module paths use importlib.import_module(), while .py entries are imported by file path — it does not recursively scan submodules.

To discover tasks in myapp.tasks.scraping, either list it explicitly:

app.discover_tasks([
"myapp.tasks",
"myapp.tasks.scraping",
"myapp.workflows",
])

Or export the decorated functions from myapp.tasks.__init__.py so importing myapp.tasks triggers the decorator registration.

Terminal window
horsies worker myapp.config:app --profile default --loglevel=INFO