Pull Request Conventions

Before Opening PR or Committing

Run make check to ensure lint, format, ty, verifytypes, and test suites pass.

For significant changes, check if documentation needs updating. This includes changes to CLI commands, new features, or workflow modifications. Update the relevant docs in the docs/ folder as needed.

For branch stability and release expectations, see the Development and Release Process. In short, PRs merge into main, but main is the active development branch rather than the stable release channel.

While Making PR

If you are an AI, always create PRs as drafts (gh pr create --draft).

PR Description

Write detailed pull request descriptions and update them when you push new changes to be correct.

Include relevant runs and trajectories in PR descriptions. When your changes involve evaluation runs or trajectories, always link to them to provide context for reviewers:

Runs: https://equistamp.linuxarena.ai/runs/<run_id> (e.g., https://equistamp.linuxarena.ai/runs/6a429632c3784f538be602f79be822fc)
Trajectories: https://equistamp.linuxarena.ai/trajectories/<trajectory_id> (e.g., https://equistamp.linuxarena.ai/trajectories/rate_limiting%3Aeavesdropper%3Ac63195_epoch1_attack_700aa832)

PR Naming

We follow Conventional Commits. We squash-merge and GitHub uses the PR title as the squash commit subject, so the PR title is the commit that lands on main — and it feeds the auto-generated release notes. Write it accordingly.

Format: type(scope): description

Type (required) — the kind of change:

Type	Use for
`feat`	A new feature or capability
`fix`	A bug fix
`docs`	Documentation only
`refactor`	A code change that neither fixes a bug nor adds a feature
`perf`	A performance improvement
`test`	Adding or correcting tests
`chore`	Tooling, dependencies, config, maintenance
`ci`	CI / workflow changes
`build`	Build system or packaging
`revert`	Reverting a previous change
`research`	Research experiments and analysis (see the blue/red PR templates)

Append ! after the type or scope to mark a breaking change: feat(eval)!: drop legacy task-set format.

Scope (optional) — the control-tower subsystem the change touches. Omit it for cross-cutting work. Common scopes: cli, eval, monitor, protocols, trajectories, mtgen, sabotage-eval, sandbox, fleet, internet-simulator, environments, viewer, live, safety, hawk. Research PRs use blue or red.

Description — imperative and concise, no trailing period.

Examples:

feat(eval): add --tags option to ct run eval
fix(internet-simulator): stream full upstream response through forward proxy
fix(trajectories): exclude None-scored side tasks from failure analysis
refactor(fleet): simplify EC2 resume path
docs: update running-evals guide
research(blue): side-task visibility experiment

The pr-title-lint workflow validates the type only; scope is unrestricted, so new subsystems never need a workflow change.

Labels: type and scope describe the change; the release notes are grouped from the commit subject's type(scope) by scripts/generate_release_notes.py, and labels drive filtering, so still add the matching label:

tooling - For tooling/infrastructure changes
environment - For environment-related changes
straj - For sabotage trajectory PRs
main-task / side-task - For task-related changes
bug - For bug fixes
documentation - For docs-only changes

After Opening a PR

IMPORTANT: Do not stop monitoring the PR until all status checks have passed and everything looks good.

Poll the PR periodically to check for:

Failed status checks: If CI fails, investigate and fix the issue. Push a new commit to trigger re-runs.
Review comments: Respond to all comments:
- If a fix is needed, make the change and push
- If you disagree or need clarification, respond explaining your reasoning
- If it's just an acknowledgment, a simple reply is fine
Merge conflicts: If the PR falls behind main and has conflicts, rebase and resolve them.

Continue checking until the PR is either merged or explicitly handed off to someone else.