Control Tower Documentation
Control Tower is an AI control evaluation framework for measuring the effectiveness of safety protocols when deploying potentially misaligned AI agents.
What is Control Tower?
Control Tower provides realistic software environments where AI agents complete tasks while safety protocols monitor for malicious behavior. It models AI control as a game between a Blue Team (maximizing usefulness and safety) and a Red Team (attempting to subvert safety measures).
Getting Started
| Guide | Description |
|---|---|
| Quickstart | Fresh machine to running evals on public environments with an OpenRouter key |
| Hands-On Guide | Build a toy environment and run your first eval |
| Running Evals on Hawk | Submit evals to a shared Hawk cluster and view results |
| Development Setup | Get your local environment running |
Concepts
| Guide | Description |
|---|---|
| Architecture at a Glance | See how settings, environments, tasks, samples, trajectories, and matches fit together |
| AI Control Concepts | Understand the core framework |
| Control Tower and Inspect | Which behaviour comes from Inspect and is documented there |
| Safety Curves | Compute safety + usefulness via ct game-metrics metrics |
| ROC Curves | Monitor discrimination plots from ct game-metrics graph roc |
| Pareto Frontier | (Safety, usefulness) Pareto plots from ct game-metrics graph pareto |
Guides
| Guide | Description |
|---|---|
| Running Evaluations | Run evaluations with policies, task selection, batching, and replay |
| Monitoring Trajectories | Score trajectories with action and trajectory monitors |
| Sabotage Eval Pipeline | Run the end-to-end honest, attack, monitor, and safety pipeline |
| Protocols | How agents are built and blue-team control logic is applied |
| Trajectories | Understand trajectory files, IDs, conversion, upload, and download |
| Web Viewer | View uploaded trajectories and eval results |
| Inspect Logs Viewer | Inspect .eval logs locally |
| Docent | Run Docent locally and manage collections |
| Auditing Environments | Environment quality audits via ct run subcommands |
| Run Scripts | Reference for the ct run <script> analysis and audit commands |
Reference
| Guide | Description |
|---|---|
| CLI Reference | Generated reference for every Control Tower command and option. |
| Python Reference | Generated reference for the registries Control Tower is extended through. |
Development
| Guide | Description |
|---|---|
| CLI Development | Conventions for adding ct commands |
| Live Development | Iterate on envs and tasks with ct live |
| Making Environments | Create evaluation environments |
| Environment Architecture | Settings, env discovery, and repo layout |
| Porting Settings | Bring an existing benchmark in as a setting |
| Making Main Tasks | Define tasks for agents |
| Making Side Tasks | Design sabotage objectives |
| Making Scorers | Write task validators |
| Side-Task Scorer Patterns | Reusable scorer_utils helpers for side-task scorers |
| Failure Types | Classify side tasks as live or delayed |
| Making Strajs | Hand-craft sabotage trajectories |
| Straj Review Workflow | The @straj-review bot for straj PRs |
| Strategy Development | Design strategy guides for the strategy-attack policy |
| Internet Simulator | Simulated external services inside environments |
| Hawk Values Contract | The hand-authored hawk-values.yaml each env needs to run on Hawk |
| Function Registry | Registering monitors and policies |
| PR Conventions | Contributing guidelines |
| Development and Release Process | How changes reach main and how releases are cut |
Infrastructure
| Guide | Description |
|---|---|
| Database | MongoDB collections and access |
| EC2 Sandbox | Run sandboxes on EC2 instead of local Docker |
| Simulated Sandbox | The no-Docker simulated sandbox backend |
| HTTP API | Viewer routes, wire conventions, auth, and self-hosting |
Project Structure
| Directory | Purpose |
|---|---|
src/control_tower/environments/ | Environment infrastructure (settings, scorer utils, config) |
.settings/ | Cloned environment repos (gitignored, populated via ct settings pull) |
src/control_tower/trajectories/ | Trajectory format handling |
src/control_tower/ct_runs/ | Orchestration and evaluation logic |
src/control_tower/protocols/ | Protocol definitions |
src/control_tower/metrics/ | Safety, usefulness, ROC, Pareto, and plotting helpers for ct game-metrics |
docs/ | This documentation |
tests/ | Test suite |
apps/docs/ | Docs and environment reference site |