Observability

JASIL owns three append-only tables. This page covers what goes in them, how to create them, and how to stop them growing forever.

Table	Written by	Purpose
event_log	The bus and the publish facade	One row per event, recording its lifecycle.
event_outbox	The publish facade	Staged events awaiting relay.
processing_jobs	The relay and the worker	One row per (event, subscriber).

The event log

jasil_settings.configure(
    jasil_settings.JasilSettings(
        event_log=jasil_settings.EventLogSettings(enabled=True),
    )
)

Off by default, because it writes to your database.

Each row records what was published, whether it was processed, by which worker, how long it took, and why it failed:

Status	Meaning
published	Written, dispatch not finished.
queued	Handed to the durable job queue (terminal here; execution is tracked in processing_jobs).
processing	A consumer picked it up.
completed	Handlers succeeded.
failed	A handler raised.
dead_letter	Abandoned after retries.

Best-effort by design

Every write opens its own short-lived session and swallows any storage error. A database hiccup degrades observability rather than breaking event processing — the trail is never the source of truth.

Because the durable route bypasses the bus, an event handed to the outbox is recorded queued explicitly. Otherwise the dashboard would go dark exactly when durable jobs are switched on.

event_log_crud.get_event_log_summary(db) gives counts by type and status, recent failures, and throughput over a window.

Migrations

The tables ship as packaged Alembic revisions behind the migrations extra:

import jasil.orm as jasil_orm
from jasil import migrations

jasil_orm.map_models(Base)  # the metadata must exist first
migrations.upgrade(engine)  # create or upgrade JASIL's tables

Function	Use
upgrade(engine)	Create or upgrade to head. Run at deploy time.
downgrade(engine, "base")	Drop JASIL's tables.
stamp(engine)	Mark an existing database as at head, without running anything.
head_revision()	The newest revision shipped in this package.
db_revision(engine)	What the database currently records.
verify_schema_current(engine)	Raise unless the database is at head.

verify_schema_current is a useful fail-fast at startup — it turns "forgot to migrate" into a clear message at boot rather than a confusing query error later.

It will not touch your tables

JASIL's migrations use their own version table, jasil_alembic_version, so they never collide with your Alembic history. Every operation is scoped to JASIL's three tables, so autogenerate cannot propose dropping yours — even though both live in the same registry.

Already created the tables with create_all?

migrations.stamp(engine)

This records head without running the baseline, so future upgrade() calls apply only genuinely new revisions.

Prefer one unified history?

Point your own env.py at your Base.metadata and add JASIL's versions directory to your version_locations. The self-contained runner above needs no host wiring, but it is not mandatory.

Retention

All three tables are append-only, so they grow without bound. Pruning runs on a schedule:

jasil_settings.configure(
    jasil_settings.JasilSettings(
        event_log=jasil_settings.EventLogSettings(retention_days=30),
        jobs=jasil_settings.JobSettings(retention_days=30),
    )
)

from jasil.retention import prune_expired_records

prune_expired_records()  # schedule daily

The two windows are independent, and <= 0 disables either.

What is deleted, and what never is

Deleted	Kept
Any event_log row past the window	Unrelayed event_outbox rows — pending work
Relayed event_outbox rows	pending / claimed jobs — in flight
completed jobs	dead_letter jobs — human-actionable

Every event_log row is prunable regardless of status: it is a safe-to-lose observability trail, and nothing in it is a source of truth. The job tables are the opposite — pruning an in-flight row would silently drop derived work, so status is checked on every delete.

Deletes run in bounded batches, each committed separately, so a prune pass never holds locks on a hot table long enough to block the relay or a worker. There is also a cap on batches per pass; a pathological backlog is drained across several passes rather than blocking the scheduler.

Pruning takes the platform lock, so only one replica does the work — the deletes are idempotent, but duplicating them is pointless load.

Logging

JASIL uses logging.getLogger(__name__) throughout — no configuration, no adapter. Records propagate to whatever handlers you have set up, and structured fields arrive via extra:

{"event_type": ..., "event_id": ..., "subscriber": ..., "event_metadata": {...}}

Loggers are named after their module (jasil.publisher, jasil.jobs.runner, jasil._core.network, …), so you can raise or lower verbosity per subsystem.

One worth routing deliberately: jasil._core.network logs at INFO every time the SSRF allowlist permits a private destination. That is an audit trail — it tells you what the exception is being used for.