Chapter 20½: inside jev-http/src — a road, a clock and a ledger

Four files. lib.rs is the front door (Jev), and the other three are what it is built from: the road (transport.rs), the clock (runtime.rs) and the meter (ledger.rs).

flowchart TD
  jev["lib.rs: Jev"] --> client["jev_client::Client"]
  jev --> ledger["ledger.rs: Ledger"]
  client --> transport["transport.rs: HttpsTransport"]
  client --> runtime["runtime.rs: TokioRuntime"]
  ledger --> file[("$XDG_STATE_HOME/jev/")]

The files, in reading order

1. lib.rs: Jev, the client you keep

• KEY is "TYPESAFE_API_KEY", the variable the key arrives in.
• Jev::from_env(binary, model) reads the key, opens the ledger with Ledger::open, and builds a client for the real API. It returns None when the key is absent or blank.
• Jev::new(key, binary, ledger, model, endpoint) is the same with every part given, which is how a test points it at jev-mock and a sandbox ledger. It builds the rustls config: compiled-in webpki-roots, any PEM from Endpoint::trusting, the RustCrypto provider, ALPN h2.
• with_guards(guards) swaps the environment's Guards for others.
• ask(state, questions, why) is the sequence drawn in Chapter 20: admit, ask through jev_client::Client, then settle, or release only when ClientError::nothing_billed says so. An out-of-credit API answer also writes the ledger's out-of-credit flag. A ledger write that fails after an answer arrived is logged to stderr and the answer is still returned: the request was already paid for.
• Answered adds to jev-client's: took (the whole round trip), the body as text, every header with credentials redacted (any name containing authorization, cookie, token, secret, -key or api-key shows as «redacted»), the request id, and a RateLimit.
• Failure is Throttled { why, retry_in }, Budget(why) or Client(ClientError). The first two never touch the network.
• RateLimit { limit, remaining, reset_seconds } is read from the conventional x-ratelimit-* and ratelimit-* headers. The vendor documents none of them on a successful response (digest §10), so it is usually None; it exists so that the day the vendor sends one, it is reported with no call site changed. last_rate_limit() returns the last one seen, and the ledger keeps a copy in last-rate-limit.json.

2. transport.rs: HttpsTransport and Endpoint

• Endpoint::api() is jev_protocol::ENDPOINT; Endpoint::parse(url) accepts any https URL and refuses anything else; trusting(pem) adds a CA file.
• HttpsTransport keeps one HTTP/2 connection warm and reopens it when it has closed, has been idle 300 s (Cloudflare closes idle connections at 400 s), or the client distrusts it.
• Connecting tries each resolved address in turn, giving every address but the last 2 s, so one blackholed address (a broken IPv6 route) cannot spend a whole attempt. It sends an HTTP/2 PING after 20 s without a frame and closes the connection if the PING goes 10 s unanswered. Flow-control windows are adaptive, because hyper's fixed windows are smaller than an 8 MiB answer.
• A response body over 8 MiB is TooLarge. A request hyper hands back unsent, or one refused with REFUSED_STREAM or cut by a server GOAWAY, is NotSent; any other break mid-exchange is Interrupted. A TLS refusal by rustls, or a server that does not negotiate h2, is Tls.
• admit waits on two per-process token buckets under the vendor's documented ceilings, 1,200 requests a minute and 250,000 tokens a second, counting the body's characters as tokens.

3. runtime.rs: TokioRuntime

now and wall_clock from std, sleep and timeout from tokio, and jitter from the clock's nanoseconds, which is random enough to spread retries and needs no dependency.

4. ledger.rs: Ledger, the shared meter

Its doc comment explains the design (why a file, why flock, which guards heal by themselves, why the total is re-read every time); read it first. Then:

• Ledger::open() uses $XDG_STATE_HOME/jev, or ~/.local/state/jev; Ledger::at(dir) uses any directory. Either creates it and, the first time ever, seeds spend.jsonl with one estimated opening line (OPENING_ESTIMATE_USD, $0.1517). create_new makes the seed race-free.
• Each line of spend.jsonl is JSON: an Entry (a real cost, or the opening estimate) or a Hold (a question admitted and not yet settled), told apart by the hold's hold_id. A line that fails to parse, such as a torn write from a killed process, is skipped with a message on stderr rather than failing every later read.
• admit(now, guards, worst_case, binary, reason) checks the out-of-credit flag, then, under one flock, reads every line into a Window, asks Window::verdict, and appends the Hold. settle(hold, entry) books the real cost; release(hold, now, why) books nothing but still counts the question.
• Window is the recent past in one pass: lifetime_usd, the admission times of the last minute, the amounts of the last hour, and the count of unsettled holds. bucket_empty_for and hour_closed_for say how long until each guard would admit.
• Guards::from_env() reads JEV_MAX_QUESTIONS_PER_MINUTE (30) and JEV_MAX_DOLLARS_PER_HOUR (0.25).
• status(now, guards) builds a Status, with line(), stopped() and stopped_for_good() for display. Its hour-breaker figure is for an ordinary question of a thousand input tokens.
• mark_out_of_credit writes out-of-credit.json (the first reason is kept), credit_restored removes it, and out_of_credit reads it.
• now() is seconds since the epoch as an f64. Every guard function takes now as an argument instead, so the tests run on any clock they like.

Try it

cargo test -p jev-http

The ledger's tests each get their own directory under the system temp dir, so they never touch your real ledger.

For the people who maintain it

In this folder

← Previous: Chapter 20: jev-http · Up: jev-http · Next: Chapter 20¾: jev-http/tests →