Database

We persist evaluation artifacts in MongoDB so they can be replayed, inspected, and surfaced in dashboards.

The CLI never needs MONGOURI

The ct CLI has no direct MongoDB access at all — there is no pymongo in control_tower. Every read and every write goes through the viewer's HTTP API, authenticated with a per-user token: run ct login to cache one, or set CONTROL_TOWER_API_TOKEN (the same token for reads and writes). MONGOURI lives only in a viewer deployment's own server environment and in the ops/ operator tooling — it is never distributed to clients.

Uploads (ct run eval, ct runs make, ct run monitor, ct traj upload) need only the API token — no MONGOURI, no AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY. The client asks the data host to presign an add-only S3 PUT, so the host holds the S3 credentials on the client's behalf. Reading a .eval back is a direct S3 GET and does use your own AWS credentials.

Read-only and local-only work touches no remote service at all: ct run eval --no-upload, ct run monitor <file> --no-upload, ct run rerun-traj <traj_like> --no-upload, ct runs make --save-dir <dir>, ct live commands, and ct run side-task-review all operate on local files, and ct traj download / ct traj list read through the website API.

How uploads reach the store

The data host holds the S3 and Mongo credentials and mediates every write:

  • POST /api/eval-logs/upload plans an add-only upload for a .eval file (a single presigned PUT) and rejects with 409 if the key already exists, or 413 if size_bytes exceeds S3's 5 GB single-PUT limit — the largest .eval object observed in the corpus is 0.17 GB, so multipart upload isn't implemented.
  • POST /api/trajectories/register writes the run header and/or trajectory documents. The server HEAD-verifies every referenced .eval object exists in S3 first, rejects partial id overlap with a 409 while an all-exist repost is an idempotent no-op ({"ok": true, "already_registered": true}), and writes run + trajectories in one Mongo transaction with no-overwrite, S3-then-Mongo ordering. Dataset locks are not checked here — registration carries no membership; the lock gate lives on the dataset publish path. See HTTP API — register semantics for the full contract.

This is the only write path into the store; clients hold no Mongo credential and issue no direct writes.

Connection Details

  • Connection: MONGOURI configures the viewer deployment's server environment and the ops/ tooling — the only Mongo clients. The ct CLI reads none of these; it holds no Mongo connection.
  • Collection names are explicit, never defaulted: MONGO_DB is the one name with a default (linuxbench); every collection env var — TRAJECTORY_COLLECTION, RUN_COLLECTION, DATASETS_COLLECTION, DATASET_VERSIONS_COLLECTION, SABOTAGE_EVALS_COLLECTION — must be set. The viewer's requireCollection (packages/viewer/lib/mongo-client.ts) throws when one is unset rather than falling back to a name. The hard-fail is deliberate: a deployment silently reading or writing the wrong release's collection is worse than one that refuses to start. ops/ requires them too, from the per-deployment file named by its mandatory --env-file; it has no defaults either, and prints the names it resolved before acting.
  • Replica set required: MONGOURI must point at a replica set, not a standalone mongod. Register writes the run header and its trajectories in one Mongo transaction, and transactions need a replica set. A single-node replica set is fine — Atlas/production is already one, and the self-host compose.yml runs Mongo with --replSet rs0 and connects with ?replicaSet=rs0&directConnection=true. Against a standalone deployment register silently degrades to sequential, non-atomic writes, so a register interrupted mid-write can leave trajectories without their run header.
  • Trajectories collection: Mongo stores a metadata document per trajectory, keyed by trajectory_id. Each document carries the pointer to the source of truth — eval_log_url, eval_sample_id, and epoch — plus the cheap filter/display fields the viewer queries (environment/main-task/side-task identifiers, success flags, monitor scores, tags, timing). Full trajectory bodies (actions, prompts, tool output) are never written to Mongo; they live in the .eval log in S3 and are read directly from there. Documents are built by control_tower.trajectories.mongo_docs.build_trajectory_document.
  • run_id is a soft reference, and run-less trajectories are a supported shape. A trajectory's run_id is a client-generated key, not a foreign key the store enforces: it may name a run header that does not exist. ct traj upload deliberately registers trajectories with no run header (upload_trajectory in agent_runs/io.py posts run=None), so their run_id resolves to nothing. This is legitimate — nothing force-empties run_id (that would break separate-run uploads) and nothing auto-creates a run header to satisfy the reference. Readers treat the reference as soft: a trajectory whose run_id does not resolve renders and lists normally (the trajectory-detail "Run" link points at /runs/<run_id>, which is a plain 404 when there is no such run), and runs are only ever enumerated from the runs collection, never synthesized from a trajectory's run_id, so a dangling run_id never produces a phantom run.
  • Runs collection: agent_runs.io.upload_run (via the register API) produces a run document containing the run_id, optional human-readable name, timestamps, the URL of the uploaded .eval log, and a metadata blob summarizing the environment, tasks, policy, and model settings. Runs also store a top-level producer field (for example ai_eval, monitor, or sample) so the website can filter agent and monitor runs efficiently, the set of trajectory_ids created for that evaluation, and a created_by field. created_by (like created_at) is server-derived: the register route stamps it from the authenticated API token's username and strips any client-sent value, so a client cannot set or spoof it. UPLOAD_USER_NAME no longer reaches any server-side record; it only labels locally-built trajectory objects.
  • Ingestion workflow: upload_run is the only client-side upload entry point. It uploads the run's .eval to S3 through the data host's presigned-upload API, then calls POST /api/trajectories/register in bounded batches to write the run header and one metadata document per sample without exceeding the data host's request-size limit. The run header rides on the final batch, so an interrupted earlier batch can leave only supported run-less trajectories rather than a header that promises missing members. Each batch retains the endpoint's idempotent retry behavior. When no .eval exists yet (post-hoc monitoring, sampling, ad-hoc uploads), one is generated from the in-memory trajectories and uploaded first, so every Mongo document is always backed by a .eval in S3.
  • Indexes: Index creation is an out-of-band operator step, never triggered by the app or CLI. An operator provisions them per cluster from the ops/ package: ct-ops setup install on a new deployment, ct-ops setup indexes to repair drift, and otherwise as the last step of ct-ops release upgrade. Index specs are frozen per release in ops/ct_ops/index_sets/.
  • Local caches: We mirror remote state locally for quick iteration. Trajectory bundles cache in ~/.cache/control-tower/trajectory-bundles/, run metadata in ~/.cache/control-tower/runs/; downloaded trajectories default to data/traj/ and run exports to data/run_trajs/. Use the helpers in control_tower.trajectories.io and control_tower.agent_runs.io instead of manipulating these directories manually so the cache stays consistent with Mongo.

Publishing Trajectories to the Website

We publish evaluation output to the website viewer so the broader team can browse runs. There are three common touchpoints: upload as part of an eval script, upload a saved .eval log by hand, and curate the run once it shows up in the UI.

Authentication setup:

  • For uploading trajectories and runs: run ct login to cache a per-user token (or set CONTROL_TOWER_API_TOKEN in your .env). Nothing else — no MONGOURI, no AWS keys (see "The CLI never needs MONGOURI" above).
  • For downloading trajectories: the same per-user token.
  • For uploading eval logs: nothing beyond that token.
  • For reading a run's .eval: AWS credentials that can GET the eval-log bucket. Runs uploaded before the S3 migration instead carry an eval_log_url on the eval-log viewer, and reading those needs LOGS_AUTH_PASSWORD — ask the linuxarena maintainers for the current value.

Upload while a script runs

  • All eval entry points upload by default: they upload the .eval log to the viewer and create runs in the database as soon as the evaluation finishes. For example:

    • uv run ct run eval ... (uploads eval and creates run)

    This needs only a per-user API token (ct login or CONTROL_TOWER_API_TOKEN). Use --no-upload to disable automatic uploading.

Upload a saved .eval

  • If the run already completed, use the CLI wrapper: uv run ct runs make logs/<file>.eval. By default this uploads the eval file (best effort), creates a run document, and uploads derived trajectories — all through the API with just the per-user token.

Add descriptions and tags in the UI

  • The website stores each run with optional name and description, so you can add human-readable summaries directly from the website once the upload lands.
  • The Control Tower upload path includes rich metadata (environment, main task, side task, models, scores, etc.), which is exposed as filterable facets. These operate as the "tags" you see in the UI, and you can extend them by editing metadata in the web interface.

Guidance on Assigning Tags

  • When building main tasks, we're required to submit a trajectory which passes the main task's scorer. We need to use the qa tag for this trajectory.
  • Similarly, after building side tasks, we're later required to build at least one sabotage trajectory for each side task. We need to use the qa tag for these sabotage trajectories.
  • For either of the above trajectories, they may have been generated by a human or an AI. We should add the ai and human tags accordingly.
  • For all sabotage trajectories (which attempt a main and side task), use the straj tag.

Uploading Sabotage Trajectories (strajs)

For creating and uploading human sabotage trajectories, see Making Strajs.

Tip: the eval logs are saved locally with --no-upload; later you can upload/create from a saved file using ct runs make <path> once you have inspected the results.

Terms

See Trajectories for the trajectory data model and how ct run, ct runs, and ct traj relate.