jevsnes.git / apps / jevprobe / README.md
1# Chapter 13: jevprobe, one question, out loud
2
3When the window's Jev panel says "off" or a choice falls back, there are
4four different things that could be wrong, and from inside the window they
5all look the same: the key never reached the process, TLS failed, the
6request was refused, or the response was not the shape the code expected.
7`jevprobe` separates them by asking one real question and printing
8everything.
9
10```bash
11op-env-run -- nix develop -c buck2 run //apps/jevprobe:jevprobe
12```
13
14It asks a single `Choice`, four things Link could do in a made-up position
15(just out of bed in his house, the staircase to the south), and prints the
16model, the round trip and how many attempts it took, the tokens, the cost,
17the confidence, every option's probability as a little bar chart, and last
18the raw response body.
19
20> **Aside: why the raw body?** The response is checked against
21> [research/jev-api-digest.md](../../research/jev-api-digest.md), a digest
22> of the vendor's documentation. A field the digest does not know about is
23> visible in the raw body and nowhere else, and finding one is a reason to
24> correct the digest.
25
26`--headers` asks the same question but prints every response header
27(anything credential-shaped redacted), the request id, the parsed rate limit
28and the token usage instead. It exists to check what the vendor's docs do
29not say (the digest's §10). Either way it is one real, billed request; run
30it sparingly.
31
32> **Try it.** Run it with the key, and once without: with no
33> `TYPESAFE_API_KEY` it says so and exits, having sent nothing.
34
35## For the people who maintain it
36
37### In this folder
38
39| Path | What |
40| --- | --- |
41| [src/main.rs](src/main.rs) | The probe: the made-up state, the `Choice`, the printout, `--headers`. |
42| [BUCK](BUCK) | The `rust_binary`, over `decisions` (for the pinned model, `decisions::model()`), `jev-protocol`, `jev-http`, `tokio` and `serde_json`. |
43
44`TYPESAFE_API_KEY` must be in this process's environment; `op-env-run` is
45what puts it there. The request goes through `jev-http`, so it is admitted
46by the same shared spend ledger as the window's questions and counted there.
47
48← Previous: [Chapter 12, native/](../native/) · Up: [apps](../) · Next: [Chapter 14, replay-probe/](../replay-probe/) →