answer-cache.tsJev's answers, kept for an hour in KV (the CACHE binding) so a request the relay has already forwarded is answered again without spending the site's key: a song whose state repeats, a page reloaded, the art check on a song someone just checked.
A request is its exact bytes. The key is the SHA-256 of the body as the relay would forward it, under the pinned model, so two requests share an answer only when every byte of what TypeSafe would read is the same; a space, a key order or one digit of the state apart is another key. Only the body is hashed: the relay forwards nothing else of the caller's (not a header, not the address), so nothing else can change the answer, and the key holds nothing of the caller either.
Only a 200 is kept, and only when it is at most MAX_BYTES and arrived whole; a refusal, an error or TypeSafe asking to back off never is. The answer is stored after the page has it (ctx.waitUntil), so a miss is not slower than no cache. A failing cache is logged and skipped: the relay then forwards as if there were none.
24type Meta = { contentType: string }; 25 26export async function cacheKey(model: string, body: ArrayBuffer): Promise<string> { 27 const digest = new Uint8Array(await crypto.subtle.digest('SHA-256', body)); 28 const hex = Array.from(digest, (b) => b.toString(16).padStart(2, '0')).join(''); 29 return `${PREFIX}/${model}/${hex}`; 30} 31 32const failed = (op: string, e: unknown) => 33 console.error({ event: 'jev.relay.cache', op, error: e instanceof Error ? e.message : String(e) });
The kept answer for key, or null.
36export async function cached(cache: KVNamespace, key: string): Promise<{ body: ArrayBuffer; contentType: string } | null> { 37 try { 38 const { value, metadata } = await cache.getWithMetadata<Meta>(key, 'arrayBuffer'); 39 if (!value) return null; 40 return { body: value, contentType: metadata?.contentType ?? 'application/json' }; 41 } catch (e) { 42 failed('get', e); 43 return null; 44 } 45}
Reads stream (a tee of the answer the page is getting) and keeps it
under key, unless it runs past MAX_BYTES or breaks off.
49export async function keep(cache: KVNamespace, key: string, stream: ReadableStream<Uint8Array>, contentType: string) { 50 const reader = stream.getReader(); 51 const chunks: Uint8Array[] = []; 52 let size = 0; 53 try { 54 for (;;) { 55 const { done, value } = await reader.read(); 56 if (done) break; 57 size += value.byteLength; 58 if (size > MAX_BYTES) return void (await reader.cancel()); 59 chunks.push(value); 60 } 61 } catch (e) { 62 return failed('read', e); 63 } 64 const body = new Uint8Array(size); 65 let at = 0; 66 for (const chunk of chunks) { 67 body.set(chunk, at); 68 at += chunk.byteLength; 69 } 70 try { 71 await cache.put(key, body, { expirationTtl: TTL_S, metadata: { contentType } satisfies Meta }); 72 } catch (e) { 73 failed('put', e); 74 } 75}
How many answers are kept now, for the data tab (data.ts): the keys under
this cache's prefix, counted (never read), at most pages KV lists of
1000; complete says whether that was all of them. A key is a request's
hash under the model, so a count is all there is to show.
81export async function cacheEntries(cache: KVNamespace, pages = 1): Promise<{ entries: number; complete: boolean }> { 82 let entries = 0; 83 let cursor: string | undefined; 84 for (let i = 0; i < pages; i++) { 85 const list = await cache.list({ prefix: `${PREFIX}/`, cursor }); 86 entries += list.keys.length; 87 if (list.list_complete) return { entries, complete: true }; 88 cursor = list.cursor; 89 } 90 return { entries, complete: false }; 91}