reference · oto feedback --help at v0.6.1

oto feedback

record what a reader said about an answer, and make it a test

oto feedback [-h] [--project PROJECT] [--question QUESTION]
                    [--verdict {wrong,incomplete,unsupported,right}] [--by BY] [--at AT]
                    [--given GIVEN] [--expected EXPECTED] [--source SOURCE] [--note NOTE]
                    [--gold GOLD] [--id ID] [--type TYPE]
                    [{record,list,promote}]

Verbs

record adds one entry
list shows them
promote turns them into gold questions

Options

Option	What it does
`--project PROJECT`	project root (default: current directory)
`--question QUESTION`	the question that was asked
`--verdict {wrong,incomplete,unsupported,right}`	wrong, incomplete, unsupported or right
`--by BY`	who is saying this (required: feedback needs an author)
`--at AT`	date, YYYY-MM-DD (default: today)
`--given GIVEN`	the answer the system gave
`--expected EXPECTED`	the right answer; without it this cannot become a test
`--source SOURCE`	a source the answer should have cited (repeatable)
`--note NOTE`	anything else worth keeping
`--gold GOLD`	gold set to promote into (default: <project>/gold/questions.jsonl)
`--id ID`	promote only these feedback ids (repeatable)
`--type TYPE`	gold question type for promoted entries

Where to use it

A reader says an answer was wrong.

oto feedback record --project claims --question "What is the collision deductible?" --verdict wrong --by "R. Osei" --given "500" --expected "750 since 2026-04-01" --source 2026-04-01-collision-deductible

an entry with an author and a date

Make it a test.

oto feedback promote --project claims

every entry with an expected answer becomes a gold question the next bench run measures

As the terminal prints it

usage: oto feedback [-h] [--project PROJECT] [--question QUESTION]
                    [--verdict {wrong,incomplete,unsupported,right}] [--by BY] [--at AT]
                    [--given GIVEN] [--expected EXPECTED] [--source SOURCE] [--note NOTE]
                    [--gold GOLD] [--id ID] [--type TYPE]
                    [{record,list,promote}]

positional arguments:
  {record,list,promote}
                        record adds one entry; list shows them; promote turns them into gold
                        questions

options:
  -h, --help            show this help message and exit
  --project PROJECT     project root (default: current directory)
  --question QUESTION   the question that was asked
  --verdict {wrong,incomplete,unsupported,right}
                        wrong, incomplete, unsupported or right
  --by BY               who is saying this (required: feedback needs an author)
  --at AT               date, YYYY-MM-DD (default: today)
  --given GIVEN         the answer the system gave
  --expected EXPECTED   the right answer; without it this cannot become a test
  --source SOURCE       a source the answer should have cited (repeatable)
  --note NOTE           anything else worth keeping
  --gold GOLD           gold set to promote into (default: <project>/gold/questions.jsonl)
  --id ID               promote only these feedback ids (repeatable)
  --type TYPE           gold question type for promoted entries