Evaluation Runs

agent_runs is the run-management layer for Control Tower. A run is the metadata container around one evaluation batch:

  • run header metadata (run_id, name, policy, task set, tags, uploader, timestamps)
  • optional eval-log URL
  • associated trajectories generated by that run

The ct runs ... command group manages saved run artifacts and metadata. It converts eval logs into run models, uploads and downloads run headers and trajectories, and provides query helpers for persisted runs.

Uploading Results

# Upload during eval
uv run ct run eval --policy attack --env web_scraping --main-task crawl_depth --side-task visit_malicious_website

# Upload with tags for easier filtering
uv run ct run eval --policy honest --env web_scraping --main-task crawl_depth --tags "experiment-1" --tags "baseline"

# Upload a previously saved eval
uv run ct runs make logs/my-eval.eval

If no explicit output flag is provided, ct runs make defaults to creating or uploading a run. It also supports writing eval logs with --save-path and writing trajectory files with --save-dir.

Required for uploads: a per-user API token — run ct login to cache one, or set CONTROL_TOWER_API_TOKEN. That is the only client credential; the CLI uploads through the viewer's HTTP API and needs no MONGOURI or AWS keys. See Database and the HTTP API reference.

Listing Runs

uv run ct runs list                      # List recent runs
uv run ct runs list --limit 10           # Limit results
uv run ct runs list --tag baseline       # Filter by tag
uv run ct runs list --format json        # Output as JSON for scripting

Inspecting a Run

uv run ct runs info <run_id>             # Show detailed info
uv run ct runs info <run_id> --format json

Sampling Runs

uv run ct runs sample <run_id> 10
uv run ct runs sample <run_id> 10 10 _ --stratified --seed 1

ct runs sample samples trajectories from a run, optionally partitioning them into non-overlapping parts. Use a trailing _ placeholder as shorthand for --keep-rest.

Downloading Runs

uv run ct runs download-trajs <run_id>   # Download trajectories for a run
uv run ct runs download-trajs https://basharena-data.vercel.app/runs/<run_id>
uv run ct runs download-eval <eval_url>  # Download an eval log by URL

ct runs download-trajs accepts a run ID from the configured API/MongoDB or a viewer run URL. See Downloading from the Viewer for viewer URL support across runs, trajectories, and datasets.