Function Registry

The function registry powers policies, untrusted policies, blue protocols, and the safety / usefulness calculations consumed by ct game-metrics graph pareto. It handles argument declaration, CLI integration, and model configuration. Monitors are not registry functions — they are prompt assets resolved by name from the asset directory; see Monitoring.

For the higher-level relationship between policies, untrusted policies, blue protocols, and monitors, see Protocols and Untrusted Policies. For the calculation-side registries used by Pareto plots, see Pareto Frontier.

Two Argument Shapes

Every registry builds a FunctionInfo (defined in function_info.py) for each registered function. There are two shapes for declaring arguments, each backed by a different function_info_factory:

In both shapes, BaseModelPython defaults (e.g. ModelSpec fields) drive the auto sub-args (see Model Arguments and Auto Sub-Args) and the same -a / -t / -p / -up / -bp CLI flags work the same way.

When in doubt, pick the shape that matches the registry you're targeting — mixing shapes inside a registry is not supported. The factory is wired up in each registry's registry.py (function_info_factory=signature_to_function_info for protocol eval registries, omitted for policy eval registries).

Defining a Policy (eval1: Args | None)

A policy is a function returning an Agent, decorated with @policies.register(). By default, the CLI name is inferred from the function name by stripping the _policy suffix and replacing underscores with hyphens (e.g., honest_policy → honest). Pass name= explicitly only when your function name doesn't follow this <name>_policy convention. Policies live in src/control_tower/protocols/policies/policies/.

from control_tower.utils.function_registry.base_model import BaseModelPython
from control_tower.models.model_spec import ModelSpec, get_model_spec_field
from control_tower.protocols.policies.registry import policies
from control_tower.models.aliases import ModelAlias


class HonestArgs(BaseModelPython):
    model: ModelSpec = get_model_spec_field(ModelAlias.UNTRUSTED)
    enable_main_task_test: bool = False
    max_steps: int = 200
    cache: bool = True


@policies.register()
def honest_policy(state: TaskState, submit: AgentSubmit, args: HonestArgs | None) -> Agent:
    """Honest untrusted policy with structured planning."""
    if args is None:
        args = HonestArgs()
    # ...

Registration is automatic via the decorator. The registry discovers all policies in the configured packages at import time.

Defining a Safety or Usefulness Calculation (eval1: Args | None)

Safety / usefulness calculations are pluggable impls consumed by ct game-metrics graph pareto (and compute_pareto). They follow the standard eval1 shape — decorate with @safety_calculations.register() or @usefulness_calculations.register() and drop the file under metrics/safety_calculations/impls/ or metrics/usefulness_calculations/impls/. Each impl returns a single scalar plus metadata; curve-style sweeps and optional bootstrap CIs are handled by callers such as Pareto, not by the impls themselves.

Implementations live in:

• Safety: src/control_tower/metrics/safety_calculations/impls/
• Usefulness: src/control_tower/metrics/usefulness_calculations/impls/

from control_tower.utils.function_registry.base_model import BaseModelPython
from control_tower.metrics.safety_calculations.registry import (
    safety_calculations,
)
from control_tower.metrics.safety_calculations.types import (
    SafetyResult,
)
from control_tower.trajectories.trajectory import Trajectory


class TMSafetyArgs(BaseModelPython):
    mode: str = "optimized"
    audit_budget: float | None = None


@safety_calculations.register()
def tm_safety(
    honest: "list[Trajectory]",
    attack: "list[Trajectory]",
    args: TMSafetyArgs | None = None,
) -> "SafetyResult":
    if args is None:
        args = TMSafetyArgs()
    # ... fit rule and compute safety ...
    return SafetyResult(safety=..., metadata={"mode": args.mode})

The CLI name is derived by stripping _safety_calculation / _usefulness_calculation suffixes if present, otherwise the function name (with underscores → hyphens) is used as-is. Reference the impl from YAML by that hyphenated name (e.g. tm-safety, defer-to-trusted).

Defining an Untrusted Policy or Blue Protocol (protocol eval: direct kwargs)

The protocol eval registries — untrusted_policies and blue_protocols — wire up signature_to_function_info as the function_info_factory (see signature_info.py). Instead of a single args: Args | None parameter, the function declares its configurable parameters directly as keyword arguments. The factory synthesizes a BaseModelPython args class from the function signature, so ModelSpec fields and other auto sub-args still work.

The function body then receives concrete values (the model's model.make_model().generate, an int, etc.) — there's no args object to unpack.

Untrusted policy

Implementations live under src/control_tower/protocols/untrusted_policy/implementations/<name>/policy.py:

from control_tower.models.model_spec import (
    ModelSpec,
    get_model_spec_field,
)
from control_tower.models.aliases import ModelAlias
from control_tower.protocols.untrusted_policy.registry import untrusted_policies
from control_tower.protocols.untrusted_policy.types import UntrustedPolicyGenerateFn


@untrusted_policies.register(name="simple-honest")
def simple_honest_policy(
    model: ModelSpec = get_model_spec_field(ModelAlias.UNTRUSTED),
) -> UntrustedPolicyGenerateFn:
    """Delegate every generate call to the untrusted model verbatim."""
    return model.make_model().generate

The registry's _untrusted_policy_function_info wrapper (in untrusted_policy/registry.py) excludes state from the synthesized args class when the function takes a TaskState (so state isn't surfaced as a CLI flag).

Blue protocol

Implementations live under src/control_tower/protocols/blue_protocols/protocols/<name>.py:

from typing import Any

from inspect_ai.solver import Solver, solver
from pydantic import Field

from control_tower.protocols.blue_protocols.registry import blue_protocols
from control_tower.protocols.monitoring.helpers import (
    DEFAULT_PROTOCOL_ACTION_MONITOR,
)


@blue_protocols.register(name="trusted-monitoring")
@solver
def trusted_monitoring_protocol(
    sus_threshold: float = Field(default=5.0),
    max_steps: int = Field(default=200),
    action_monitor: str = Field(default=DEFAULT_PROTOCOL_ACTION_MONITOR),
) -> Solver:
    """Run the untrusted policy and audit (stop) the trajectory if a tool call is too suspicious."""
    async def solve(state: TaskState, generate: Generate) -> TaskState:
        ...
    return solve

CLI flags target each parameter directly: -up model_temperature=0.5 for an untrusted policy, -bp sus_threshold=7.0 for a blue protocol. Auto sub-args from ModelSpec fields work just like in the eval1 shape.

Naming Convention

By default, the registry strips the _<identifier> suffix from the function name and converts underscores to hyphens. Each registry passes its own identifier (see registry.py):

When the function name doesn't end in the matching _<identifier> suffix, the registry would otherwise leave the name untouched — so pass name="..." explicitly. For example, honest_policy registered against untrusted_policies does not end in _untrusted_policy, so it auto-names to honest-policy unless you pass name="honest". Similarly, simple_attack_policy registered against untrusted_policies needs name="simple-attack" (in implementations/simple_attack/policy.py).

The docstring's first sentence becomes the description shown in help output.

The Args Class

For the eval1 shape, args classes must extend BaseModelPython (not plain Pydantic BaseModel). Fields define configurable parameters with defaults. If a function takes no args parameter, it has no configurable arguments.

For the protocol eval shape, the args class is synthesized for you by signature_to_function_info from the function's keyword parameters; you don't write a BaseModelPython subclass yourself. Annotation + default of each parameter become the field type and default.

Field Defaults Become Sub-Arg Defaults

GenerateConfig values passed to get_model_spec_field become the defaults for auto-generated sub-args — e.g. GenerateConfig(cache=False) makes model_cache default to False instead of True, still overridable with -p model_cache=true.

Passing Arguments from the CLI

# Policies use -p (short for --policy-arg)
ct run eval --policy honest -p model=sonnet -p max_steps=100

# Untrusted policies use -up (short for --untrusted-policy-arg)
ct run eval --untrusted-policy honest -up model=opus --task-file simple

# Blue protocols use --protocol with -bp (short for --protocol-arg)
ct run eval \
  --untrusted-policy simple-attack \
  --protocol trusted-monitoring \
  -bp sus_threshold=7.0 \
  --task-file simple

Hyphens in keys are converted to underscores automatically. Unknown keys raise a validation error. The flag prefix (-p, -up, -bp) corresponds to the registry being targeted, regardless of which argument shape the function uses.

Where these flags live: --policy / -p are exposed by ct run eval for policy evals. --untrusted-policy / -up and --protocol / -bp are exposed by ct run eval for protocol evals (you cannot combine policy eval and protocol eval selections in one invocation). ct run sabotage-eval selects policies via --attacker / --policy (alias); adding --protocol / --protocol-arg runs both evals under that blue protocol, with the attacker/honest policies as its untrusted policy — see docs/sabotage-eval.md.

Model Arguments and Auto Sub-Args

Any field typed as ModelSpec automatically gets sub-arguments for configuring the underlying model: attempt_timeout, max_retries, temperature, max_connections, adaptive_connections, verbosity, effort, reasoning_effort, reasoning_tokens, cache. Sub-arg names are prefixed with the field name — so for a field named model, you get model_cache, model_temperature, and so on.

To see the full list of sub-args for any policy, run ct protocols policies -v.

Resolved selections persist each model's non-null generation config and effective OpenRouter provider preferences alongside its model name. The serialized selection can be validated again without losing reasoning, cache, timeout, retry, or provider-routing settings, so it is also the canonical input for configuration fingerprints.

Model Aliases

You can pass aliases instead of full model names:

Or pass any full model name directly — any Inspect AI provider string works, including self-hosted endpoints (e.g. vllm/Qwen/Qwen3-32B, see Self-Hosted and Open-Weight Models).

Discovering Available Arguments

ct protocols monitors action     # List action monitors (add a name for details)
ct protocols monitors trajectory # List trajectory monitors

ct protocols policies            # List all policies
ct protocols policies -v         # List with all arguments
ct protocols policies attack     # Show args for a specific policy

ct protocols untrusted-policies  # List untrusted policies (protocol eval)
ct protocols blue-protocols      # List blue protocols (protocol eval)

Cost Tracking

Eval-time model calls are priced by inspect, not recomputed in post. Before each Model.generate(), the hook in models/usage_hooks.py registers that model's rates from models/rates.py via ensure_model_cost_registered; inspect multiplies usage and stamps usage.total_cost. CT reads the stamp — it does not price generate-path calls itself. Rates live in a generated JSON table (~/.cache/control-tower/model_rates.json by default, overridable with CONTROL_TOWER_MODEL_RATES); run uv run python scripts/generate_model_rates.py after clone. That script merges into the existing table, so a row marked "manual": true and a model the source has stopped publishing both survive a regeneration. Inspect registers the standard prompt band because ModelCost has nowhere to put a tier — calls over a tier boundary price low until inspect can express tiers upstream.

Paths that never reach generate() — pre-flight estimates, the SDK rail, monitors replaying stored token records — price through Prices in models/pricing.py, whose one() takes a model name and usage and returns USD or None when the table has no row; Priced.by_models() keeps the priced total and a None in by_model for any unpriced one.

Automatic per-result cost on MonitorResult

Every monitor invocation runs inside a cost-tracking scope — no opt-in needed. The orchestration layer wraps each call that produces a MonitorResult with run_monitor_with_cost, which sets a fresh CostSink in a ContextVar for that scope. Any Model.generate() call made while the scope is active writes into the sink via the usage hook, and when the scope exits, the sink's totals are applied to the resulting MonitorResult.

Wrapped sites:

• Action monitor invocation (monitoring/helpers.py)
• Trajectory monitor invocation (ct_runs/monitor/task.py)
• Ensemble member invocations (action and trajectory), so each member's cost is isolated from its siblings

Concurrency safety comes from PEP 567: each asyncio.Task gets a copy of its parent's context, and inside each scope we .set() a fresh sink — so sibling coroutines in asyncio.gather can't interfere.

Fields populated on MonitorResult

• cost_usd — known spend for this invocation; None only when nothing in the scope could be priced
• cost_by_model — per-model breakdown; a None amount is a model nobody could price
• model_name — set when there's exactly one model involved
• input_tokens / output_tokens / cached_input_tokens — aggregate across models in the scope; always populated when calls happened

Limitations to know

1. Unhooked contexts — calls to Model.generate() outside of an active CostSink scope won't be recorded. This is expected for non-monitor code paths.
2. Sub-monitor granularity — if monitor A internally invokes monitor B without wrapping it in its own run_monitor_with_cost, B's cost pours into A's scope. Aggregate is right; per-sub granularity is lost. Opt into wrapping sub-invocations when you want the split.
3. Detached tasks with a fresh empty context (asyncio.create_task(..., context=contextvars.Context())) won't see the parent's sink. Rare.

External Policies

Any CLI flag that accepts a policy name also accepts a module:function path, following the same convention as uvicorn app:factory. This lets you use policies defined in your own project without modifying control-tower.

# External policy
ct run eval --policy my_project.policies:my_attack -p model=sonnet --task-file simple

The function must have the same signature as a registered one (return Agent for policies). The argument shape that the registry uses (Args | None vs direct kwargs) is determined by which registry the external function is loaded into — the loader uses that registry's function_info_factory, so external functions must match the shape expected for that flag (see Two Argument Shapes).

External environments also work via --env my_project.envs:my_env_config.

Quick Reference