Chapter 17: jev-protocol — the grammar book that never posts a letter

jev-protocol is the TypeSafe System One protocol (POST /v1/systemone) with no network, no files and no clock. It builds the exact bytes of a request, checks that a response really answers the questions that were asked, classifies the API's errors, and decides whether and when to retry. Something else does the sending. This crate only knows what a correct letter looks like and how to read the reply.

Three kinds of question

Jev never writes prose back. You hand it a state (whatever it is judging: a string, an object, an array) and one or more named questions, and every answer is a probability. There are three shapes of question:

The limits are checked when a question is built: Choice::new refuses one option, 256 options, an empty label or a repeated one, and Score::new refuses one level or eleven. A request that breaks an API rule cannot be constructed, so it is never sent.

A request is one JSON object, written in this order:

{"model":"jev-1.13.0","questions":{"q":{"type":"noul","instructions":"Urgent?"}},"state":{"body":"hi","id":1}}

That line is the exact output of the exact_bytes test in src/request.rs.

Why the bytes matter so much

postjevsql keeps a cache of Jev's answers keyed by a hash of what was asked, and lmjtfy keeps an archive keyed by the exact request body. Both only work if the same question always produces the same bytes. So:

• state goes on the wire byte for byte as the caller gave it (Json).
• Json::canonical writes a value in RFC 8785 form: sorted keys, no whitespace, ECMAScript number formatting. A number a double cannot hold exactly (a bigint past 2^53, a numeric(38,20)) becomes a string with every digit, so nothing is rounded away and equal values give equal bytes.
• Choice options are sent in the caller's order and never pass through a map type that sorts. Order is part of the question: Jev's answers show an order bias.
• question_bytes gives one question exactly as it appears inside the request, for a cache key that covers a single question.

Aside. Generic JSON canonicalizers parse numbers into f64 and silently round. That is why src/jcs.rs is hand-written: it keeps every number's original text until it knows a double can hold it.

Reading the answer back

flowchart LR
  Q["Questions<br/>noul / choice / score"] -->|"request_bytes"| B["exact bytes"]
  B -->|"a transport sends them"| W(("Jev"))
  W -->|"200 body"| P["Response::parse"]
  P -->|"Key&lt;NoulAnswer&gt;"| A["get(key).noul"]
  W -->|"any other status"| E["ApiError::from_response"]

Adding a question returns a Key<A>, typed by the kind of answer it reads (Key<NoulAnswer>, Key<ChoiceAnswer>, Key<ScoreAnswer>). It is the Rust version of a TypeScript generic that makes answer.choice a compile error on a yes-or-no question. A key belongs to the Questions that issued it; using it on another set's response panics.

Response::parse refuses a reply that:

• comes from a different model than the pinned one (ModelMismatch), so a moved model is never recorded under the pinned version;
• leaves a question unanswered, or answers one that was not asked;
• answers a question as another type;
• picks a Choice label that was not offered, or leaves one without a probability;
• gives a probability or confidence outside [0, 1], or a score outside its levels.

What passes is ordered like the question: a Choice's probabilities come back in the options' order, a Score's lowest level first.

Model ids, errors, retries and the price

• ModelId::pinned takes <name>-<major>.<minor>.<patch>, with an optional -suffix (jev-1.13.0, jev-2.0.0-rc1). It refuses aliases (jev-latest, jev-preview) because the vendor says an alias moves when a new release ships.

• ApiError::from_response reads a non-200 body in all three shapes the API sends ({"detail": {"error_type", "message"}}, a list of field errors, or anything else, quoted up to 500 characters) and classifies it:

  Status	ApiErrorKind
  400 with error_type max_tokens_exceeded	ContextOverflow (split the request; never resent)
  400	BadRequest
  402, or 403 whose message mentions "credit", "balance", "insufficient funds" or "quota exceeded"	OutOfCredit (inferred; the vendor documents no such response)
  401, other 403	Authentication
  404, 408, 422, 429, 529	NotFound, RequestTimeout, Unprocessable, RateLimited, Overloaded
  other 5xx	Server

• retry is the vendor SDKs' policy (Python typesafe-sdk 0.7.1, JS @typesafe-ai/sdk 0.6.0): at most 2 retries, 10 s per attempt, 30 s for everything. 408, 429 and every 5xx are retried, and so is a request that got no response. The wait is retry-after-ms, else Retry-After in seconds or as an HTTP date, capped at 60 s; without either it is 0.5 s doubling to 5 s, less up to 25% jitter. next_delay takes the jitter and the wall clock as arguments, so the policy itself has no clock.

• The price is DOLLARS_PER_MTOK, $0.042 per million input tokens; output tokens are free. Usage::dollars prices an answer. worst_case_dollars prices a request before it is sent: its length in characters (no token is shorter than one character), capped at the documented 64,000-token ceiling, times the 3 attempts the retry policy may make.

Try it

cargo test -p jev-protocol

Besides the unit tests, never_panics in src/lib.rs feeds random bytes to the four parsers that meet the network (Response::parse, ApiError::from_response, retry::next_delay, Json::canonical): they must reject, never panic.

This crate builds for wasm32-unknown-unknown as it is: it has no I/O to port.

For the people who maintain it

In this folder

Design points taken from prior art, both MIT OR Apache-2.0: typed answer keys, limits checked before sending, responses verified against the questions and structured API errors from JedimEmO/typesafe-client; fuzzed parsers from kunobi-ninja/kunobi-jev. Unlike both, the state is sent as caller-supplied bytes, so the bytes on the wire are exactly the bytes a cache key hashes.

← Previous: Chapter 16: jevcrates · Up: jevcrates · Next: Chapter 17½: inside jev-protocol/src →