1# Chapter 18: jev-client — the patient one
2
3`jev-client` asks Jev a question and keeps at it: it retries what is worth
4retrying, gives each attempt its own time limit, resends a request that
5never left, and puts the request id on every error. It has no network and no
6clock of its own. It reaches the world through two small interfaces, called
7*ports*, which whoever uses it fills in.
8
9## The problem with "just call fetch"
10
11Sending one HTTP request is easy. Sending it correctly when the network is
12flaky takes rules, and the rules have to be the same everywhere, or one
13project ends up paying for a question twice because it resent something the
14others knew not to. Here are the ones this crate keeps:
15
16- A `429` or a `503` is worth another go; a `403` is not.
17- A request that never reached the server can be resent at once, for free.
18- A request that broke halfway may already have been billed, so resending
19  it is a real retry, counted and backed off.
20- Nothing waits forever: 10 seconds per attempt, 30 seconds for the lot.
21
22Those rules are `jev-protocol`'s retry policy. This crate is the loop that
23applies them, and only this crate has one.
24
25## Ports: the loop does not care which road it takes
26
27```mermaid
28flowchart LR
29  C["Client::ask"] -->|"send one request"| T{{"Transport"}}
30  C -->|"now, sleep, timeout, jitter"| R{{"Runtime"}}
31  T --- H["jev-http: HttpsTransport"]
32  T --- W["jev-worker: FetchTransport"]
33  T --- P["postjevsql-pg: a Postgres backend"]
34  T --- F["the tests: a scripted fake"]
35  R --- HR["TokioRuntime"]
36  R --- WR["WorkerRuntime"]
37  R --- FR["a virtual clock"]
38```
39
40A **`Transport`** sends one `HttpRequest` and returns the whole
41`HttpResponse`, or a `TransportError` whose `TransportErrorKind` says what
42happened (`NotSent`, `Connect`, `Interrupted`, `Tls`, `TooLarge`, `Config`).
43It may also wait in `admit` before an attempt, for a rate limit shared with
44other senders, and it is told to `distrust_connection` after a timeout.
45
46A **`Runtime`** supplies `now`, `wall_clock`, `sleep`, `timeout` and
47`jitter`. In the tests, `sleep` just moves a counter forward, so a case that
48waits 30 simulated seconds finishes in microseconds.
49
50If you have written a JavaScript function that takes `fetch` as an argument
51so a test can pass in a fake, this is that idea, with the clock passed in as
52well.
53
54> **Aside.** The futures these traits return need not be `Send`. That is for
55> postjevsql: its transport runs inside one Postgres backend, tied to one
56> thread, and a `Send` bound would have shut it out.
57
58## One ask, start to finish
59
60```mermaid
61flowchart TD
62  start["ask(state, questions)"] --> bytes["request_bytes"]
63  bytes --> left{"budget left?"}
64  left -->|"no"| timedout["TimedOut"]
65  left -->|"yes"| admit["transport.admit<br/>(wait not charged to the budget)"]
66  admit --> send["send, under min(10 s, budget left)"]
67  send -->|"200"| parse{"Response::parse"}
68  parse -->|"ok"| done["Answered"]
69  parse -->|"wrong"| resp["Response error"]
70  send -->|"NotSent, under 2 redials"| left
71  send -->|"Tls / TooLarge / Config"| fail["Transport error"]
72  send -->|"400 max_tokens_exceeded"| api["Api error, not resent"]
73  send -->|"other status, Connect, Interrupted, timeout"| delay{"next_delay<br/>and still inside 30 s?"}
74  delay -->|"no"| giveup["Api / Transport / TimedOut"]
75  delay -->|"yes"| sleep["sleep, add x-typesafe-retry-count"]
76  sleep --> left
77```
78
79A few details the picture leaves out:
80
81- A redial is the same attempt sent again on a fresh connection: it does not
82  count as an attempt, does not back off, and sends no retry-count header.
83  Each ask allows two; a third `NotSent` is treated as a broken exchange and
84  retried.
85- A timed-out attempt makes the client call `distrust_connection`, so the
86  next attempt does not reuse a connection that may be stuck.
87- Every request carries `Authorization: Bearer <key>`, marked sensitive so it
88  stays out of HPACK's dynamic table and out of `Debug` output, and
89  `Content-Type: application/json`. A retry adds `x-typesafe-retry-count`,
90  as both vendor SDKs do.
91- An optional **`Observer`** hears each `Event` as it happens: `Sending`,
92  `Retrying` (with cause, delay and request id), `Redialing` and `Answered`
93  (with attempts and token usage). `client.observed(observer)` attaches one.
94
95## What comes back
96
97`Ok(Answered)` holds the verified `Response`, the `request_id`, the number
98of `attempts`, and the answering response's raw `headers` and `body`, for a
99caller that wants something the parse does not read.
100
101`Err(ClientError)` is one of `Invalid` (nothing was sent), `Api` (the API
102refused, after any retries), `Transport`, `TimedOut`, or `Response` (a 200
103that does not answer the questions). Every variant that saw a response
104carries its request id, the only handle for a billing dispute.
105`nothing_billed()` is true only when nothing can have cost money: the request
106was never built, or its one attempt was refused outright. A spend guard uses
107it to decide whether to release what it held for the request.
108
109## Try it
110
111```
112cargo test -p jev-client
113```
114
115Fifteen tests, every one on virtual time: the server's `retry-after-ms`
116honoured to the millisecond, three `503`s backing off 0.5 s then 1 s, a
117stalled attempt costing exactly 10 s, and three stalls spending exactly the
11830-second budget. The scripted fake is in Chapter 18¾.
119
120Using it takes a transport, a runtime, a pinned model and a key:
121
122```rust
123let client = jev_client::Client::new(transport, runtime, model, &key)?;
124let answered = client.ask(&state, &questions).await?;
125```
126
127## For the people who maintain it
128
129### In this folder
130
131| Path | What |
132| --- | --- |
133| [src/](src/) | The code, file by file: Chapter 18½. |
134| [Cargo.toml](Cargo.toml) | Dependencies: `bytes`, `http`, `jev-protocol`, `web-time`. |
135| [CLAUDE.md](CLAUDE.md) | Invariants for agents. |
136
137← Previous: [Chapter 17½: inside jev-protocol/src](../jev-protocol/src/) · Up: [jevcrates](../) · Next: [Chapter 18½: inside jev-client/src](src/) →