Trajectories

trajectories is the canonical trajectory data and conversion layer for Control Tower.

A trajectory is a structured record of agent behavior for a sample:

  • environment/main-task/side-task context
  • action/tool-call history
  • timing/cost/monitor metadata
  • optional run linkage (run_id) and eval linkage (eval_log_url, sample ID)

This package handles:

  • core data model + validation (trajectory.py)
  • conversion from eval logs/tool calls/paths/IDs (convert.py, from_eval_log.py, from_tool_calls.py)
  • storage and retrieval (filesystem/API helpers in io.py, api.py) and Mongo document building (mongo_docs.py)
  • analysis utilities (analysis.py)
  • trajectory-focused CLI commands (cli.py)
  • TUI support for interactive trajectory review/edit (in control_tower.live.tui)

CLI Overview

Top-level command group:

  • ct traj ... (wired from control_tower.trajectories.cli:traj_cli)

Primary commands:

  • ct traj make <TRAJ_LIKE> [--sample-id ID] [-e ENV] [-t MAIN_TASK] [-s SIDE_TASK] [--save-path PATH]

    • Converts trajectory-like input into one trajectory JSON.
    • TRAJ_LIKE can be a file path, directory, URL, trajectory ID, run ID, or eval-like input.

  • ct traj list [--fs] [-e ENV] [-t MAIN_TASK] [-s SIDE_TASK] [--tag TAG] [--limit N]

    • Lists trajectories from the API (default) or the local filesystem (--fs).

  • ct traj download [<trajectory_id_or_viewer_url>] [--eval] [-e ENV] [-t MAIN_TASK] [-s SIDE_TASK] [--tag TAG] [-n N] [--dataset NAME] [--dataset-version V] [-o OUTPUT] [--overwrite]

    • Downloads one trajectory (by ID or viewer URL); bulk-downloads the latest N matching trajectories when filter options are passed instead of an ID (max 500, default 20); or downloads every trajectory in a dataset with --dataset NAME (no count limit).
    • --eval downloads the trajectory's full .eval archive instead of the bundle JSON.
    • --dataset fetches the dataset's full trajectory ID list, then downloads those trajectories by ID in batches, so the whole dataset arrives completely (no auth or Mongo URI required) instead of truncating on a single oversized stream. Add --dataset-version V to download a specific version (default: latest).
    • Accepts a trajectory ID from the configured API/MongoDB or a viewer trajectory URL (e.g. https://basharena-data.vercel.app/trajectories/<id>). See Downloading from the Viewer.

  • ct traj upload <path_to_trajectory_json>

    • Uploads a local trajectory JSON to MongoDB.

  • ct traj patch <traj> ...

    • Generates a patch from a trajectory using live tooling.

  • ct traj apply <eval_log_file> ...

    • Applies file edits from an eval log into an environment codebase.

  • ct traj tui <trajectory_path> ...

    • Opens the trajectory TUI for interactive review/editing.

  • ct traj top-sus <TRAJ_LIKE> [--limit N] [--min-score X] [--env ENV] [--full] [--json]

    • Shows top suspicious actions using monitor scoring attached to actions.

Cost fields on a Trajectory

Every cost on a converted Trajectory is inspect's stamp on usage.total_cost. Sample totals come from sample.model_usage; per-call figures come from each ModelEvent. An Inspect response-cache read is left out of model_usage (actual spend) and marked on timing.model_calls[i].inspect_cache_hit. Provider prompt-cache discounts are already in the stamp.

  • Per-call: action.timing.model_calls[i].cost_usd.
  • Agent total: Trajectory.agent_cost is {model: usage.total_cost} from sample.model_usage.
  • Legacy aggregate: Trajectory.cost is sum(sample.model_usage.values(), ModelUsage()).
  • Structured breakdown: cost_breakdown.traj_task_cost carries total_usd / by_model. A null in by_model is an unpriced model; the viewer keeps the known total and marks it $N+.

Monitor costs live on each MonitorResult.cost_usd (see Cost Tracking in function-registry.md). Helpers on Trajectory:

  • trajectory.monitor_cost() -> Priced — per-model sum across action and trajectory monitor results. Ensemble-safe: walks members rather than double-counting the parent.
  • trajectory.total_cost() -> Priced — agent + monitor, per model. Priced carries by_model, where a None amount is a model nobody could price, and derives total_usd from the ones it could.
  • trajectory.total_cost_usd() -> float | None — known spend, or None only when nothing at all was priced. A None in by_model is what says the total is short.

Cost tracking is automatic; MonitorResult.cost_usd=None means nobody could price that call. A None in by_model is how the viewer tells the total is short and marks it $N+.

How trajectories fits with other CLI groups

  • ct run ... runs experiments and can produce trajectories.
  • ct runs ... manages run-level containers around groups of trajectories.
  • ct traj ... is the direct trajectory manipulation and analysis surface.

Typical workflows

Convert input to a trajectory JSON:

  • uv run ct traj make logs/sample.eval -o data/traj/sample.json

List trajectories from API:

  • uv run ct traj list -n 20

Download one trajectory:

  • uv run ct traj download <trajectory_id>
  • uv run ct traj download https://basharena-data.vercel.app/trajectories/<id> (viewer URL)
  • uv run ct traj download <trajectory_id> --eval (full .eval archive instead of bundle JSON)

Bulk-download the latest matching trajectories:

  • uv run ct traj download -e web_scraping --tag baseline -n 100

Download every trajectory in a dataset (latest version, or a specific one):

  • uv run ct traj download --dataset LaStraj
  • uv run ct traj download --dataset LaStraj --dataset-version 2

Inspect suspicious actions:

  • uv run ct traj top-sus data/run_trajs/<run_id> --limit 15