jevstrudel.git / website / src / jev / ask.mjs
ask.mjsannotatedask.mjssource136 lines · 6.3 KB · raw

One call to Jev from the page, through this site's relay (RELAY; worker/src/relay.ts), which adds the site's TypeSafe key and forwards it to api.typesafe.ai. TypeSafe's API refuses browser origins (CORS), so a page cannot call it directly, and visitors never need a key of their own. The relay forwards only the question shapes the page asks; widen it there first when a new one is needed.

7export const JEV_MODEL = 'jev-1.13.0';
8export const RELAY = '/jev/v1/systemone';

The page's own timing, for the relay's log (worker/src/relay.ts): how many seconds before its section begins each request went out, and which attempt it was. Headers, not body fields, so the body the relay forwards is exactly what TypeSafe reads; the relay sends TypeSafe none of the page's headers. Listeners heard answers land after their section began while headless runs could not make one, and these say which side is slow.

16export const LEAD_HEADER = 'Jev-Section-In';
17export const ATTEMPT_HEADER = 'Jev-Attempt';

On every answer the relay gives: hit when it came from the relay's answer cache (the same request, byte for byte, answered within the hour; TypeSafe not asked), miss when TypeSafe answered it (worker/src/answer-cache.ts). The network panel shows it per call.

23export const CACHE_HEADER = 'Jev-Cache';

On answers to a signed-in listener (worker/src/relay.ts): their daily Jev budget as used/limit on each call it charged, and spent on the 429 that refuses one once it is used up (with Retry-After until 00:00 UTC, which the back-off below honours, so songs play their fallbacks until then). account.mjs listens, to keep the header's count current.

30export const BUDGET_HEADER = 'Jev-Budget';
31const budgetListeners = new Set();
32export function onBudget(listener) {
33  budgetListeners.add(listener);
34  return () => budgetListeners.delete(listener);
35}

A budget header from a call made elsewhere (sandbox.mjs, for code the listener's own AI played), told to the same listeners.

38export function noteBudgetHeader(value) {
39  if (value) for (const listener of budgetListeners) listener(value);
40}

Signing in or out changes whose budget and which rate limit a call is charged to, so a back-off earned by the other one no longer applies.

44export function resetBackoff() {
45  blockedUntil = 0;
46  budgetSpent = false;
47  failures = 0;
48}

Backing off as TypeSafe's docs ask: after a 429 or 529 (or a 503 that says when to come back), every call on this page waits: for the response's retry-after-ms or Retry-After when it has one (as the SDK honours them), else exponentially from 0.5 s to 5 s with a quarter of jitter (the SDK's defaults). Until then a call fails at once, without a request, so songs play their fallbacks and the mood picker, radio and art check say why.

57let blockedUntil = 0;
58let budgetSpent = false; // the back-off in force is the daily budget's
59let failures = 0;
60const BACKOFF = { initialMs: 500, maxMs: 5000, jitter: 0.25 };

62function waitMs(res) {
63  const ms = Number(res.headers.get('retry-after-ms'));
64  if (Number.isFinite(ms) && ms >= 0 && res.headers.has('retry-after-ms')) return ms;
65  const value = res.headers.get('Retry-After');
66  if (value !== null) {
67    const seconds = Number(value);
68    if (Number.isFinite(seconds)) return seconds * 1000;
69    const date = Date.parse(value); // an HTTP date
70    if (Number.isFinite(date)) return Math.max(0, date - Date.now());
71  }
72  const exp = Math.min(BACKOFF.initialMs * 2 ** failures, BACKOFF.maxMs);
73  return exp * (1 - Math.random() * BACKOFF.jitter);
74}
75
76const backsOff = (res) =>
77  res.status === 429 || res.status === 529 || (res.status === 503 && res.headers.has('Retry-After'));

A call that fails in transit (no connection, or a 5xx that is not TypeSafe asking us to back off: the relay restarting, a dropped upstream) is asked again, twice, 1 s then 2 s later. A Jev call has no side effects, so asking twice is safe, and a section is seconds away.

83const RETRY_MS = [1000, 2000];
84const transient = (res) => res.status >= 500 && !backsOff(res);
85const pause = (ms) => new Promise((r) => setTimeout(r, ms));

How a call reaches the relay: the page's fetch, or, inside the sandbox a listener's song plays in (sandbox/player.mjs), a message to the page, which makes the call without the visitor's session (sandbox.mjs).

90let relayFetch = (url, init) => fetch(url, init);
91export function setRelayFetch(f) {
92  relayFetch = f;
93}

secondsToSection() is read as each attempt goes out: a retry is later.

96async function post(body, secondsToSection) {
97  for (let attempt = 0; ; attempt++) {
98    const headers = { 'Content-Type': 'application/json', [ATTEMPT_HEADER]: String(attempt) };
99    const lead = secondsToSection?.();
100    if (Number.isFinite(lead)) headers[LEAD_HEADER] = lead.toFixed(2);
101    try {
102      const res = await relayFetch(RELAY, { method: 'POST', headers, body });
103      if (!transient(res) || attempt >= RETRY_MS.length) return res;
104    } catch (e) {
105      if (attempt >= RETRY_MS.length) throw e;
106    }
107    await pause(RETRY_MS[attempt]);
108  }
109}

Resolves to the answers, keyed like questions. secondsToSection, when given, returns how long until the section asked about begins (or null when the song is not playing), for the relay's log.

114export async function askJev(questions, state, { secondsToSection } = {}) {
115  const wait = blockedUntil - Date.now();
116  if (wait > 0) {
117    if (budgetSpent) throw new Error('your daily Jev budget is spent; it resets at 00:00 UTC');
118    throw new Error(`backing off; Jev is asked again in ${Math.ceil(wait / 1000)} s`);
119  }
120  const res = await post(JSON.stringify({ model: JEV_MODEL, questions, state }), secondsToSection);
121  const budget = res.headers.get(BUDGET_HEADER);
122  noteBudgetHeader(budget);
123  if (backsOff(res)) {
124    const ms = waitMs(res);
125    failures += 1;
126    blockedUntil = Date.now() + ms;
127    budgetSpent = budget === 'spent';
128    if (budgetSpent) throw new Error('your daily Jev budget is spent; it resets at 00:00 UTC');
129    const why = res.status === 429 ? 'rate limited' : 'overloaded';
130    throw new Error(`${why}; Jev is asked again in ${Math.ceil(ms / 1000)} s`);
131  }
132  failures = 0;
133  if (!res.ok) throw new Error(`relay answered ${res.status}: ${(await res.text()).slice(0, 120)}`);
134  const body = await res.json();
135  return body.answers ?? {};
136}