Trajectory Prompt

Runs custom LLM prompts against trajectories for analysis.

Purpose

General-purpose tool for running arbitrary LLM prompts against trajectories. Useful for:

  • Extracting insights from trajectories
  • Summarizing agent behavior
  • Custom analysis tasks
  • Research questions about agent actions

Commands

ct run traj-prompt

Run an LLM prompt against trajectories.

Usage:

ct run traj-prompt TRAJ_LIKE [OPTIONS]

Arguments:

  • TRAJ_LIKE - Trajectory file, eval log, trajectory ID, or run directory

Options:

  • -p, --prompt - Prompt name from the prompts directory
  • -m, --model - Model to use (default: untrusted model)
  • --json - Output as JSON for scripting
  • --list - List available prompts and exit
  • -n, --name - Name for the run (used in output directory)
  • -P, --parallel - Maximum concurrent LLM requests (default: 5)
  • -L, --limit - Limit number of trajectories to process

Examples:

# List available prompts
ct run traj-prompt --list

# Run summary prompt on a trajectory
ct run traj-prompt data/traj/example.json -p summary

# Run on eval log with high concurrency
ct run traj-prompt logs/my_eval.eval -p summary -P 10

# Limit to first 5 trajectories
ct run traj-prompt logs/*.eval -p summary -L 5

ct run traj-prompt-view

View results from a traj-prompt run interactively.

Usage:

ct run traj-prompt-view [RUN_PATH]

Controls:

  • ←/→ - Navigate between results
  • e - Show only errors
  • a - Show all results
  • q - Quit

Examples:

# Select from available runs
ct run traj-prompt-view

# View specific run
ct run traj-prompt-view data/traj_prompt_runs/summary_20260201_143022/

Available Prompts

Prompts are stored in prompts/:

  • summary.md - Summarize trajectory behavior
  • egregiousness.md - Rate how egregious sabotage attempts are
  • Custom prompts can be added as needed

Output

Results saved to data/traj_prompt_runs/<prompt_name>_<timestamp>/:

  • results.jsonl - LLM responses for each trajectory
  • config.json - Run configuration
  • prompts/ - Copy of prompts used

Technical Details

  • Template-based prompting with trajectory context
  • Supports any prompt in markdown format
  • JSON output mode for programmatic use
  • Parallel processing for efficiency
  • Token usage tracking