Configuring Horsies
Prerequisites
Section titled “Prerequisites”- PostgreSQL 14+
- Python 3.13+
- libpq 17+ for a time limit on the cancel request of an interrupted query. The
horsies[pg-binary]extra bundles its own libpq. See psycopg and libpq
Installation
Section titled “Installation”uv add horsiesBasic Configuration
Section titled “Basic Configuration”from pydantic import SecretStrfrom horsies import Horsies, AppConfig, IsolatedChildren, PostgresConfig, WorkerProfile
config = AppConfig( broker=PostgresConfig( database_url=SecretStr("postgresql+psycopg://user:password@localhost:5432/mydb"), ), worker_profiles={ "default": WorkerProfile(isolated=IsolatedChildren(count=8)), },)
app = Horsies(config)worker_profiles names the settings that a worker starts with: here, one profile with 8 children. See Worker Profiles.
PgBouncer / Transaction Pooling
Section titled “PgBouncer / Transaction Pooling”If your provider gives you a transaction-pooled PgBouncer URL, keep ordinary SQL
traffic on that pooled URL and give Horsies a second direct/session-capable URL
for schema setup and LISTEN/NOTIFY.
from horsies import Horsies, AppConfig, PostgresConfig
config = AppConfig( broker=PostgresConfig( database_url=SecretStr(os.environ["DATABASE_URL_POOLED"]), session_database_url=SecretStr(os.environ["DATABASE_URL_DIRECT"]), pgbouncer_transaction_mode=True, ),)
app = Horsies(config)Managed providers, such as PlanetScale Postgres, may show separate pooled and
direct connection strings. Verify the current ports and URLs in your provider
dashboard or docs. Workers do not support a PgBouncer-only URL because
PostgreSQL LISTEN requires a persistent session.
Custom Queues with Priorities
Section titled “Custom Queues with Priorities”Different operations have different urgency levels can be defined with priority values. 1-100 where 1 is priority numero uno.
from horsies import Horsies, AppConfig, PostgresConfig, QueueMode, CustomQueueConfig
config = AppConfig( queue_mode=QueueMode.CUSTOM, custom_queues=[ CustomQueueConfig(name="urgent", priority=1, max_concurrency=10), CustomQueueConfig(name="standard", priority=50, max_concurrency=20), CustomQueueConfig(name="low", priority=100, max_concurrency=5), ], broker=PostgresConfig( database_url=SecretStr("postgresql+psycopg://user:password@localhost:5432/db_name"), ),)
app = Horsies(config) # use this app instance for decorating task| Queue | Priority | Use Case |
|---|---|---|
urgent |
1 | The most important queue |
standard |
50 | Things in between |
low |
100 | The least important, can wait |
Task Discovery
Section titled “Task Discovery”app.discover_tasks([ "myapp.tasks", "myapp.workflows",])discover_tasks records module paths for later import by the worker. Dotted module paths use importlib.import_module(), while .py entries are imported by file path — it does not recursively scan submodules.
To discover tasks in myapp.tasks.scraping, either list it explicitly:
app.discover_tasks([ "myapp.tasks", "myapp.tasks.scraping", "myapp.workflows",])Or export the decorated functions from myapp.tasks.__init__.py so importing myapp.tasks triggers the decorator registration.
Running the Worker
Section titled “Running the Worker”horsies worker myapp.config:app --profile default --loglevel=INFO