Skip to content

Data Model

All Snowpack state lives in a single Postgres database. This page documents every table, its columns, and the key indexes that support the hot-path queries.

jobs

The primary record for each maintenance request. One row per job.

ColumnTypeConstraintsDescription
job_idTEXTPKUnique job identifier
database_nameTEXTNOT NULLIceberg database name
table_nameTEXTNOT NULLIceberg table name
actionsTEXT[]NOT NULLOrdered list of maintenance actions
dry_runBOOLNOT NULLWhether to simulate without writing
statusTEXTNOT NULLOne of: pending, running, completed, failed, cancelled
submitted_atTIMESTAMPTZNOT NULLWhen the job was submitted
started_atTIMESTAMPTZWhen the worker began execution
completed_atTIMESTAMPTZWhen the job reached terminal state
errorTEXTError message if failed
attempt_idTEXTDL-197 fence — current claim’s unique ID
deleted_atTIMESTAMPTZSoft-delete tombstone
created_atTIMESTAMPTZDEFAULT NOW()Row creation timestamp

job_queue

Transient work queue. Rows exist only while a job is in-flight (pending or running). Deleted on ack.

ColumnTypeConstraintsDescription
job_idTEXTPK, FK -> jobsReferences the parent job
enqueued_atTIMESTAMPTZDEFAULT NOW()When the job entered the queue
visible_atTIMESTAMPTZNOT NULLClaim lease deadline — job becomes re-claimable after this time
claimed_atTIMESTAMPTZWhen a worker claimed the job
claimed_byTEXTWorker hostname
attempt_idTEXTFence echo — must match jobs.attempt_id
attempt_countINTDEFAULT 0Number of claim attempts

job_actions

Per-action results within a job. One row per action execution.

ColumnTypeConstraintsDescription
job_idTEXTFK -> jobsParent job
actionTEXTAction name (e.g., rewrite_data_files, expire_snapshots)
successBOOLWhether the action succeeded
errorTEXTError message if the action failed
messageTEXTHuman-readable result summary
elapsed_secondsREALWall-clock duration of the action
created_atTIMESTAMPTZRow creation timestamp

Primary key: (job_id, action, created_at)

table_locks

Per-table ownership lock. Prevents concurrent maintenance on the same table.

ColumnTypeConstraintsDescription
table_keyTEXTPK"{database}.{table}" composite key
holderTEXTNOT NULLjob_id of the lock owner
acquired_atTIMESTAMPTZDEFAULT NOW()When the lock was acquired
expires_atTIMESTAMPTZNOT NULLLock expiration deadline

table_cache_entries

Cached table inventory from the catalog. Refreshed atomically by TableCacheSyncWorker.

ColumnTypeConstraintsDescription
database_nameTEXTIceberg database name
table_nameTEXTIceberg table name
maintenance_enabledBOOLnullableWhether maintenance is enabled for this table
payloadJSONBFull table metadata payload

Primary key: (database_name, table_name)

table_cache_meta

Single-row metadata about the table cache sync state.

ColumnTypeConstraintsDescription
idINTPK, CHECK (id = 1)Single-row invariant
last_synced_atTIMESTAMPTZLast successful sync timestamp

health_snapshots

Precomputed health snapshots written by the health sync CronJob. Historical, not on the hot path.

orchestrator_runs

Audit log of orchestrator CronJob executions. Historical, not on the hot path.

Key indexes

IndexDefinitionPurpose
idx_jobs_visiblePartial index WHERE deleted_at IS NULLSupports the live-read joined view — excludes soft-deleted rows so the API’s job listing query only scans active jobs
idx_job_queue_readyComposite on (claimed_at, visible_at)Serves both KEDA’s scaler query (SELECT COUNT(*) ... WHERE claimed_at IS NULL AND visible_at <= NOW()) and the worker’s dequeue_with_claim CTE