lmjtfy.git / tools / eval / src / README.md
1# Chapter 14½: inside eval/src
2
3Two files, one per question the eval answers.
4
5- **`main.rs`**: which model is the LLM. It opens with what a run costs,
6  then `CASES` (each input with the tools a careful person would accept),
7  then the loop: build the Worker's request, send it over Cloudflare's REST
8  API with the token read from 1Password for that run, parse the reply as
9  the Worker does, and score it.
10- **`facts.rs`**: whether Jev's first request reads inputs the way the rules
11  need. 25 cases, each with the facts it should come back with, sent through
12  `jev-http` so the shared spend ledger admits it.
13
14Both spend real money or neurons, so neither runs in `cargo test`.
15
16| File | What |
17| --- | --- |
18| [main.rs](main.rs) | The LLM eval, and the entry point for `lmjtfy-eval`. |
19| [facts.rs](facts.rs) | The facts eval, `lmjtfy-eval facts`. |
20
21← Previous: [Chapter 14, eval/](../) · Up: [eval](../) · Next: [Chapter 15, ds-bundle/](../../../ds-bundle/) →