Chapter 9: archive, never pay for the same answer twice

Here is the rule, in the owner's own words: never send the same exact shape to the same model, ever. If a request has been answered before (the same bytes, to the same model) it is not sent again, by anyone, unless someone presses ↻ on it (below). The kept response comes back instead. Nothing is spent, nothing waits on the network, and the page says "not sent" and when it was answered.

This matters more than it sounds. Jev is calibrated and deterministic enough that the same question really does deserve the same answer. And the way people use a site like this is to drop a link in a chat, where a dozen people click it within a minute. Without the archive that is a dozen identical calls. With it, it is one.

sequenceDiagram
  participant A as Visitor A
  participant B as Visitor B
  participant R as Archive
  participant J as Jev
  A->>R: this request, please
  R->>R: kept? no. On the wire? no.
  R->>J: send it (once)
  B->>R: the same request, please
  R->>R: on the wire: wait for it
  J-->>R: the response
  R->>R: keep it
  R-->>A: the response (sent now)
  R-->>B: the same response (kept)

The archive is a Durable Object (chapter 2), so there is exactly one of it, and every visitor's request goes through the same one. It does not just look things up: it makes the calls itself. That is what lets two visitors asking the same new thing at once share one call, and why a call finishes and is kept even if the visitor who started it closes the tab. A call that got no response (an error, a timeout) keeps nothing, so it can be tried again.

Aside. Why key on the whole request body and not a hash of it? Because a hash can collide, and the rule is absolute. Two different requests can never be taken for one if what is compared is the requests themselves.

Try it. Open https://lmjtfy.fun/?q=Is+a+slot+machine+a+good+retirement+plan%3F, scroll to the calls under the answer, and look for "not sent". Then look at "So far" on the home page: how many requests were never sent because the same one had been answered before.

Asking again, and every answer kept

Jev is calibrated, but a model can change, and the LLM in front of it is not deterministic at all. So every call panel has ↻: it sends that request for real, and keeps the response as its newest version. Whatever comes after is worked out again from the new answer. A later request that comes out the same is served from the archive; one that changes (the LLM wrote different questions, say) is new, and is sent.

Every response a request ever got is kept, numbered from 1. A panel with more than one shows ◀ answer 2 of 3 ▶, and the newest is the default. Stepping back is only looking: it sends nothing, and neither does anything after it. If an old answer leads to a request nobody has asked yet, the page says so and offers ↻ for it rather than sending it.

flowchart LR
  P["the page's $pins: 3f2a9c01be44:2"] --> W["the Worker: what does each call want?"]
  W -->|"pinned: Version(2)"| A["archive: version 2 of that request"]
  W -->|"after a pinned one: KeptOnly"| B["archive: the newest kept, or NotKept"]
  W -->|"↻: Fresh"| C["archive: send now, keep as newest"]
  W -->|"anything else: Latest"| D["archive: the newest kept, or send"]

Aside: the page names calls by a hash, the archive does not. A panel needs a short name for its buttons, so call_id is twelve hex digits of the SHA-256 of where the request went and what it said. The archive still looks responses up by the request itself.

The feed

The same store keeps the questions people asked, and the home page shows them: asked lately and most asked, with Jev's answer beside each and how many times it was asked. Each line is a link that asks it again, for free. The feed shows nothing about who asked. (What the site keeps for its owner is another matter: chapter 2, What is kept about visitors.) A question goes on the list only if the rule "list it" holds (chapter 6), which needs Jev's fit fact: Jev moderates its own feed, and the owner can overrule it either way. An unfit question is still answered for whoever asked it, and still kept. Asked lately goes back as far as the feed does: scroll it to its end and the next fifty load. So far counts the questions, the share Jev answered with no LLM, the requests sent and not sent, and clones and pulls of the code.

For the people who maintain it

This package is the messages between the Worker and the archive object, with no I/O and no clock; the object is apps/lmjtfy/src/archive.rs, and its storage is drawn in chapter 2.

Admin is the other set of messages, for the owner's admin backend, sent to the object's /admin path by a Worker of its own: Select (one statement that only reads, checked by reads_only) and Moderate (the owner's say on a question's place in the feed). Answered is the reply.

Live is what the archive pushes to open pages over /live: Online, Toast, Patch and Build. Pins reads the page's $pins and says what each call should Pick; call_id names a call for it. Place and Seen are where an open page is, kept on its socket.

An Answer is what is kept of what Jev said: the few words the page prints large, and the numbers behind them (Detail: the probability, or the confidence with every option or level and its probability). The feed prints the words; a link preview prints and draws the numbers too. Answers kept before the numbers were are read as words alone (stored). when writes a moment the way the page shows it.

In this folder

← Previous: Chapter 8½, llm/src/ · Up: packages · Next: Chapter 9½, archive/src/ →