Chapter 10: budget, being fair to strangers with a shared wallet

The site is free to use, so it is free to abuse. Somebody could sit on it and ask questions until the money runs out, and then nobody else could ask anything. So there are two kinds of limit, and this chapter is both.

Two daily budgets, shared by everyone. One for the LLM, counted in Workers AI "neurons" (Cloudflare's unit of AI compute), and one for Jev, in dollars. They reset at 00:00 UTC. The home page shows what is left of both.

A limit per visitor. One visitor may ask 10 full questions a minute, and the as-you-type request may run 60 times a minute. Past that the page says "Slow down." and sends nothing. The count is kept in memory only, keyed by the visitor's address for that minute. (The address is kept, with the rest of each request, in the archive: chapter 2, What is kept about visitors.) It keeps one person from emptying the day for everyone; the budgets remain the hard stop.

Aside: hold, then settle. You cannot know what a call costs until it is done, but you must decide whether to make it before. So each call first holds its worst case (the most it could possibly cost), and afterwards settles to what it really cost. Two visitors arriving at the same moment cannot both spend the last of the day, because the first hold already took it.

sequenceDiagram
  participant A as Archive
  participant B as Budget object
  participant J as Jev or the LLM
  A->>B: hold the worst case
  B-->>A: held (or: the day is spent)
  A->>J: the call
  J-->>A: the response, and what it cost
  A->>B: settle at the real cost

The neuron budget belongs to the whole Cloudflare account, not just this site: the eval, a local dev server and anything else on the account spend the same 10,000. So the budget object also reads the account's own figure from Cloudflare's analytics API (at most every 30 seconds) and counts from there (observe). It is raised to that figure and never lowered by it, because the figure lags and the meter's own spending since is already on top. If the figure cannot be read, it counts its own calls alone and the page says so.

The account is on the Workers Free plan (checked 2026-10-02), where Cloudflare refuses Workers AI calls past the allocation and bills nothing. The counter is what makes the page stop cleanly and say why, and what would hold the line if the account moved to a paid plan.

Try it. The home page's "Shared budget" window, https://lmjtfy.fun, is these meters, live.

For the people who maintain it

No I/O and no clock: the arithmetic, and the messages the Worker and the budget object (apps/lmjtfy/src/meter.rs) exchange. Meter with hold, settle, observe and used; Hold and Exhausted; Which (neurons or Jev dollars) and its storage key; day and date for UTC days; Visits and VISIT_WINDOW_MS for the per-visitor count; Ask, Told, Counted and Status for the messages.

In this folder

← Previous: Chapter 9½, archive/src/ · Up: packages · Next: Chapter 10½, budget/src/ →