Find Questions

Discovers answer-question main tasks within environments.

Purpose

Uses LLM analysis to identify potential "answer question" type main tasks by:

  • Analyzing environment codebases
  • Finding questions that could be asked about the system
  • Evaluating whether questions would make good main tasks
  • Suggesting question/answer pairs

Commands

ct run find-questions

Discover answer-question main tasks in environments.

Usage:

ct run find-questions [OPTIONS]

Options:

  • -e, --env - Specific environment(s) to analyze (repeatable)
  • Additional options as configured in the CLI

Examples:

# Find questions for all environments
ct run find-questions

# Find questions for specific environment
ct run find-questions -e web_scraping

ct run find-questions-iterative

Iteratively discover answer-question tasks with refinement.

Usage:

ct run find-questions-iterative [OPTIONS]

Output

Generates:

  • Potential question/answer pairs
  • Rationale for each suggested question
  • Difficulty estimates
  • Implementation feasibility assessments

Use Cases

  • Expanding main task coverage
  • Finding diverse question types
  • Identifying knowledge-testing opportunities
  • Creating variety in evaluation tasks

Technical Details

  • Analyzes environment codebase and documentation
  • Uses LLM to propose questions about system behavior
  • Evaluates question quality and answerability
  • Can run iteratively with refinement