Skip to main content

ARCH-26: Durable background process lifecycle

Quick orientation​

Decision​

Tyrum supports durable local background processes through a separately launched, independently managed supervisor. The native Tyrum runtime remains the only conversation-turn executor. The supervisor is an ordinary local capability service: it accepts a direct executable plus ordered argument vector, owns the resulting child tree, renews the workspace lease, writes bounded redacted output, and records lifecycle state through fenced database compare-and-set transitions.

The gateway never auto-spawns the supervisor. Background tools are absent unless configuration enables them and a credential-authenticated handshake proves a supported topology. The only enabled topology in this increment is independent-service. A gateway and supervisor in the same cgroup fail closed. Windows and independently managed container topology are unsupported because no separate-container replacement test is available in the required environment. Linux deployments require measured service cgroup divergence; macOS requires an operator attestation that does not contradict the measured process topology.

ARCH-26 adds four tools:

  • process.start starts one whole-workspace, direct-argv run and returns an opaque handle.
  • process.get reads lifecycle state for a handle owned by the calling tenant, agent, and conversation.
  • process.logs reads bounded redacted output using stable source cursors.
  • process.cancel records idempotent cancellation intent for an owned handle.

All four retain ARCH-25 executable-process classification and exact-literal exposure/profile gates. Only the runtime policy shell-approval predicate treats process.get and process.logs as non-shell reads. Shared mode, wildcard-only exposure, unconfigured targets, and hardened confinement remove the full background family.

Supported topology​

The supported unit has two independently managed services:

  1. the gateway, which authenticates the model request, evaluates policy, creates the queued row, and acquires the initial fenced lease; and
  2. the supervisor, which owns accepted live runs, pipes, redaction context, retained files, child-tree termination, database state transitions, and lease renewal.

They may share a local Tyrum home and database, but not a service failure domain. SQLite is supported only on a local filesystem; network-mounted SQLite/WAL is outside the support boundary. SQLite connections use a 250 ms busy timeout, and background claim, transition, settlement, and renewal writes use immediate transactions plus bounded retries below the 15-second renewal safety margin. The reference deployment is the systemd unit in deploy/tyrum-process-supervisor.service. The Docker Compose file intentionally does not advertise a supervisor topology.

Every IPC frame uses protocol version 1, is newline-delimited JSON capped at 64 KiB, and carries the 32-byte supervisor credential. The credential is stored below the private Tyrum home, never appears in tool arguments, database rows, manifests, evidence, or logs, and is compared in constant time. Parent directories, the socket, credential, run directories, manifests, and results must have the required owner and non-group/world-writable modes; symlinks are rejected.

Lifecycle and ownership​

The lifecycle is:

The supervisor is authoritative after acceptance. Every live transition compares the tenant/run identity, current state, and monotonic state_seq; late or duplicate events are audited no-ops. Terminal states are absorbing. The first supervisor-owned result wins cancel, timeout, output-limit, and exit races.

The supervisor signals only child trees present in its in-memory ownership table. It never signals, adopts, or reconstructs ownership from a PID stored in the database. If identity or outcome cannot be proven after supervisor loss, the run becomes orphaned_unconfirmed and its workspace remains quarantined. An operator must explicitly acknowledge the unresolved liveness risk before clearing quarantine.

Reconciliation is not limited to gateway boot. Stale heartbeats trigger a bounded liveness proof during live status/handle reads and operator list/drain operations. An unreachable or instance-mismatched supervisor follows the same conservative unproven settlement path. The supervisor command also reconciles before binding a replacement socket. No recovery path reads or signals a database PID.

The whole workspace is the only supported scope. workspace_scope must be "."; subtree claims fail subtree_scope_unsupported. A process-run:<run-id> lease owner and generation fence prevent stale renewal or release. Conflicting mutators fail immediately with blocked_by_background_run; they do not wait for lease expiry. Lease-renewal failure causes the supervisor to terminate its owned tree before the safety margin and to quarantine any uncertain result.

Output and secrets​

Output is file-backed, bounded per stream and per run, and read with forward cursors or a bounded tail request. The supervisor applies streaming UTF-8 redaction before bytes reach retained files. Redaction context is capped per run and retained only for the supported run lifetime. Newly resolved values are sent through same-tenant/run/workspace/agent/conversation secrets.update frames before the resolving tool proceeds; both live sinks extend their carry window before acknowledging the update. Each update and each same-instance, scope-validated redaction.sync is audited.

Model-facing process.logs synchronizes request-scoped context without changing global engine state, reads the already-redacted file, preserves the source cursor through the dedicated buildToolContent path, then applies model-boundary sanitation and untrusted-content tagging exactly once. A missing or overflowing redaction context fails closed. After supervisor replacement, retained files may be read only through the clearly untrusted operator path until pruning; they are never silently exposed to the model.

Persistence, quotas, and retention​

Migration 168 adds:

  • process_runs, containing immutable scope/provenance, safe executable display and hashes, state, fencing, counters, quarantine, and outcome truth;
  • process_actions, containing idempotent operator/model intent and its result; and
  • process_events, containing bounded/coalesced lifecycle and denial audit.

No row or host manifest stores raw argv or environment values. Environment names are sorted and values contribute only to a request hash. process_actions and process_events have composite tenant/run foreign keys with cascade cleanup. Workspace, conversation, policy, and approval identities remain documented soft references where a proven exported parent/delete contract does not exist.

Configuration is an optional process.background object with no schema default. Absence means disabled and round-trips as absence for rollback. When enabled, it bounds concurrent runs per workspace, tenant, and agent; runtime; output; retained bytes; redaction context; terminal output; and event history. The supervisor timer owns pruning. Manual pruning and coalesced audit keep model-triggered growth bounded.

Snapshots include all three tables but never host files or redaction context. Importing onto another host marks all output unavailable. Imported non-terminal rows settle to orphaned_unconfirmed, with workspace_quarantined=false, outcome_reason=imported_on_foreign_host, and an audit event. No source-host process can exist on the target, so foreground mutation remains possible there.

Authorization and residual risk​

Handles are random opaque identifiers, not authority. Every get, log, and cancel request validates the immutable tenant, workspace, agent, and conversation scope and returns the same unknown_handle response for missing and mismatched records. Repeated ownership denials are coalesced and expire.

This design does not mitigate same-UID cross-tenant arbitrary control, read, cancellation, or secret-memory risk. A hostile process running as the same operating-system UID can access the supervisor's local credential, socket, files, process namespace, or memory. Database tenant checks and queued-row revalidation provide request attribution and replay containment only; they are not a security boundary against that attacker. Deployments needing hostile-tenant isolation require separate OS users, containers/VMs, credentials, databases, and policy boundaries outside ARCH-26.

Baseline confinement is the only supported posture for these tools. ARCH-26 does not claim hardened confinement, PID-based adoption, descendant certainty after escape, crash-proof output durability, network-filesystem SQLite safety, or Windows support.

Operations​

The operator surface provides:

  • tyrum process-supervisor to run the independent service;
  • tyrum process list plus the authenticated gateway status route to report live/quarantined rows and support posture;
  • tyrum process drain to stop new starts and settle live rows;
  • tyrum process settle to explicitly acknowledge unresolved orphan risk; and
  • tyrum process prune to run idempotent retained-output/event cleanup.

Tenant status responses expose only derived availability, configured topology, proof, and reason; raw boot/cgroup/namespace details require operator scope. The liveness probe is cached for a few seconds.

Update treats a nonexistent SQLite file as a never-booted installation. Existing SQLite and Postgres databases are probed without migrations; update refuses only when the database cannot be inspected or a live/quarantined process record remains.

Rollout​

The feature ships dark. An operator first deploys the independent supervisor, verifies the authenticated supported-topology status, enables the optional background configuration, and then adds exact tool and execution-profile entries. Canary one non-production workspace with one active run before broader use.

Upgrading an existing container volume requires its Tyrum home and ancestors to be private. Volumes previously created with group-writable 0770 permissions must be migrated to owner-only 0700 before enabling the supervisor; the runtime fails closed and never repairs unsafe ancestors.

Rollback​

Rollback is ordered:

  1. Drain the supervisor.
  2. Prove there are zero live or quarantined runs.
  3. Write a deployment revision with the entire process.background key removed and remove the four exact exposure/profile entries.
  4. Verify the pre-ARCH-26 configuration parser can boot the restored row.
  5. Revert the gateway source or replace the supervisor.

Migration 168 and its audit records remain additive after rollback.