jevstrudel.git / website / src / jev / soundPick.mjs
soundPick.mjsannotatedsoundPick.mjssource122 lines · 5.4 KB · raw

Jev picking a sound for what someone describes (the MCP's ask_jev_sound, tabTools.mjs): a Choice whose options are the sounds tab's sound names.

TypeSafe takes at most 255 options a Choice, and the tab can hold thousands of sounds. More than that are asked in banks, as Krug's JevLM does (JevLM-Open, jevlm/multipass.py): the candidates split into ceil(n / 255) banks of near-equal size, one question per bank, several questions to a request within the relay's size; each bank's few likeliest go on to one final question among them. So the final probabilities are among those finalists, not every sound, and the answer says so. More than MAX_BANKS banks is refused rather than asked: narrow first.

12import { askJev } from './ask.mjs';

14export const MAX_OPTIONS = 255; // TypeSafe's own limit for a Choice
15export const MAX_BANKS = 16; // as JevLM's MAX_VOCABULARY_OPTIONS: 4080 sounds
16export const FINALISTS_PER_BANK = 3;

a request's questions, within the relay's 64 KiB with room for the state

18export const MAX_REQUEST_BYTES = 48 * 1024;
19export const DESCRIPTION_MAX = 280;

21const INSTRUCTIONS =
22  'A musician describes the sound they want for a Strudel song (`want`). Choose the sound that fits it best, ' +
23  'judging by its name (drum machines are "<machine>_<sound>", gm_ names are General MIDI instruments, wt_ ' +
24  'are wavetables), its kind and how many variants it has.';

What Jev reads of a sound: its name, its kind, its variants.

27const describe = (s) => `${s.name}: ${s.category}${s.type && s.type !== 'sample' ? ` ${s.type}` : ''}${s.variants > 1 ? `, ${s.variants} variants` : ''}`.slice(0, 200);
28const criteriaOf = (sounds) => Object.fromEntries(sounds.map((s) => [s.name, describe(s)]));

n candidates in ceil(n / max) banks, sizes differing by at most one.

31export function banks(items, max = MAX_OPTIONS) {
32  const count = Math.ceil(items.length / max);
33  const size = Math.floor(items.length / count);
34  const extra = items.length % count;
35  const out = [];
36  let at = 0;
37  for (let i = 0; i < count; i++) {
38    const length = size + (i < extra ? 1 : 0);
39    out.push(items.slice(at, at + length));
40    at += length;
41  }
42  return out;
43}

45const choice = (sounds) => ({ type: 'choice', instructions: INSTRUCTIONS, criteria: criteriaOf(sounds) });
46const bytes = (x) => new TextEncoder().encode(JSON.stringify(x)).length;

Questions packed into requests of at most MAX_REQUEST_BYTES each.

49export function pack(questions) {
50  const requests = [];
51  let current = {};
52  let size = 0;
53  for (const [key, q] of Object.entries(questions)) {
54    const b = bytes({ [key]: q });
55    if (size && size + b > MAX_REQUEST_BYTES) {
56      requests.push(current);
57      current = {};
58      size = 0;
59    }
60    current[key] = q;
61    size += b;
62  }
63  if (size) requests.push(current);
64  return requests;
65}

The likeliest k of an answer's options, most likely first.

68function likeliest(answer, sounds, k) {
69  const known = new Map(sounds.map((s) => [s.name, s]));
70  const ranked = Object.entries(answer?.probabilities ?? {})
71    .filter(([name, p]) => known.has(name) && Number.isFinite(p))
72    .sort((a, b) => b[1] - a[1])
73    .map(([name]) => known.get(name));
74  // the chosen one first, even if its probabilities disagree
75  const chosen = known.get(answer?.choice);
76  const out = chosen ? [chosen, ...ranked.filter((s) => s !== chosen)] : ranked;
77  return out.slice(0, k);
78}

Jev's pick among sounds (soundEntry()s) for want: resolves to { picks: [{ ...sound, p }], confidence, candidates, banks, calls, scope }, scope 'all' when every candidate was in the final question, else 'finalists'. ask is askJev (tests pass their own).

84export async function pickSound(sounds, want, { ask = askJev, top = 3 } = {}) {
85  const text = String(want ?? '').trim().slice(0, DESCRIPTION_MAX);
86  if (!text) throw new Error('say what sound you want');
87  if (sounds.length < 2) {
88    if (sounds.length === 1) return { picks: [{ ...sounds[0], p: 1 }], confidence: null, candidates: 1, banks: 0, calls: 0, scope: 'all' };
89    throw new Error('no sounds match; list_sounds shows what the tab has');
90  }
91  const state = { want: text };
92  let calls = 0;
93  let finalists = sounds;
94  let bankCount = 1;
95  if (sounds.length > MAX_OPTIONS) {
96    const split = banks(sounds);
97    bankCount = split.length;
98    if (bankCount > MAX_BANKS) {
99      throw new Error(`${sounds.length} sounds are too many to ask about (at most ${MAX_BANKS * MAX_OPTIONS}); narrow with category or filter`);
100    }
101    const questions = Object.fromEntries(split.map((bank, i) => [`bank${i}`, choice(bank)]));
102    const answers = {};
103    for (const request of pack(questions)) {
104      calls++;
105      Object.assign(answers, await ask(request, state));
106    }
107    finalists = split.flatMap((bank, i) => likeliest(answers[`bank${i}`], bank, FINALISTS_PER_BANK));
108    if (finalists.length < 2) throw new Error('Jev did not answer about the sounds');
109  }
110  calls++;
111  const { sound: answer } = await ask({ sound: choice(finalists) }, state);
112  const picks = likeliest(answer, finalists, top).map((s) => ({ ...s, p: answer.probabilities?.[s.name] ?? null }));
113  if (!picks.length || picks[0].name !== answer?.choice) throw new Error(`Jev answered "${String(answer?.choice).slice(0, 64)}", which is not one of the sounds`);
114  return {
115    picks,
116    confidence: typeof answer.confidence === 'number' ? answer.confidence : null,
117    candidates: sounds.length,
118    banks: bankCount > 1 ? bankCount : 0,
119    calls,
120    scope: bankCount > 1 ? 'finalists' : 'all',
121  };
122}