lib.rsThe Jev API over HTTPS for a native process: jev-client's policy on
this crate's transport (transport.rs: HTTP/2 over pure-Rust TLS on
tokio) and runtime, and the shared spend ledger in front of both.
WHAT to ask is jev-protocol's; how to retry is jev-client's. This is
only how the bytes get there, and what they cost. The browser build cannot
reach the API directly anyway (api.typesafe.ai refuses browser
origins), so a web page gets a relay and never links this.
Every request is checked against [ledger::Ledger] before it is sent.
This account's credit is finite, so a client that could be built and used
without a shared, on-disk record of what has already been spent would make
overspending a matter of which binary you happened to run - see ledger's
own doc comment for why the record has to be a file and not a number kept
in memory. Jev::new/Jev::from_env are the only ways to get a client,
and both build the ledger check into it, so there is no path to ask that
skips it. There is no self-imposed lifetime cap here (removed 2026-09-22,
the user's own ruling) - the only hard stop is the vendor's own
out-of-credit answer, which is a fact about the account, not a guess.
25use std::sync::{Arc, Mutex, PoisonError}; 26use std::time::{Duration, Instant}; 27 28use jev_client::{Client, ClientError}; 29use jev_protocol::{ApiErrorKind, DOLLARS_PER_MTOK, Json, ModelId, Questions, Response, request_bytes, retry}; 30 31pub use ledger::{Entry, Guards, Hold, Ledger, NoCredit, Refusal, Status, Window}; 32pub use runtime::TokioRuntime; 33pub use transport::{ENDPOINT, Endpoint, HttpsTransport};
The environment variable the key arrives in. On this machine it reaches a
process through op-env-run and nowhere else - never a dotfile, never a
command line (see the repository README).
38pub const KEY: &str = "TYPESAFE_API_KEY";
The documented per-request input ceiling (digest §4), so a huge state
cannot make a worst case run away.
42const MAX_REQUEST_TOKENS: f64 = 64_000.0;
What a response's headers said about how much of the vendor's own rate
limit is left, when they said anything at all. Digest §10: as of
2026-09-20 the vendor documents no such headers on a successful response -
only Retry-After/retry-after-ms on a 429/529 - so every field here
is commonly None. It stays a typed, if usually-empty, snapshot rather
than nothing at all so that the day the vendor adds one, this starts
reporting it with no code changed at every call site.
Seconds until the window resets, exactly as the header gave it - not resolved to a clock time, since that would claim a precision about when the header was actually read that nothing here can back up.
A client, and one to reuse: it holds the connection, and a warm connection is worth hundreds of milliseconds per decision.
Which binary this is, for the ledger's records - native, player,
jevprobe, ...
The most recent parsed [RateLimit], kept for [Jev::last_rate_limit]
so a caller (the budget MCP tool, in particular) does not have to
wait for the next request to see what the last one learned.
What one answered request cost in time as well as tokens.
The answers, verified against the questions asked.
82 pub response: Response,
The whole round trip, retries and waits included.
84 pub took: Duration,
How many times the request had to be sent. 1 is the happy path.
86 pub attempts: u32,
The response exactly as it arrived. Kept so that a field the protocol does not know about is visible rather than silently dropped by the parse - which is how the digest gets corrected.
90 pub body: String,
Every response header, name and value, with anything that looks like
a credential redacted. apps/jevprobe --headers exists to print this.
93 pub headers: Vec<(String, String)>,
x-typesafe-request-id, if the response carried one (digest §1).
Why an ask produced no answer. The first two never touch the network.
A shared guard that clears itself refused: the same request would be
admitted after retry_in. The caller goes on without an answer.
104 Throttled { why: String, retry_in: Duration },
Nothing is admitted until a human acts: the vendor said the account is out of credit, or the ledger cannot be read or written (a guard that cannot see the spend fails closed).
108 Budget(String),
The request went out and failed, as jev-client judged it after its
retries.
114impl std::fmt::Display for Failure { 115 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { 116 match self { 117 Self::Throttled { why, retry_in } => { 118 write!(f, "not sent, throttled: {why}; clears in {:.0}s", retry_in.as_secs_f64()) 119 } 120 Self::Budget(why) => write!(f, "not sent: {why}"), 121 Self::Client(e) => e.fmt(f), 122 } 123 } 124} 125 126impl std::error::Error for Failure {} 127 128impl Jev {
Read the key from the environment and build a client for the API, its
ledger opened from $XDG_STATE_HOME (see ledger::Ledger::open).
None when the key is not there, which is not an error: the app runs
without it and says on screen that it cannot play.
133 pub fn from_env(binary: impl Into<String>, model: ModelId) -> Option<Result<Self, String>> { 134 let key = std::env::var(KEY).ok().filter(|key| !key.trim().is_empty())?; 135 Some( 136 Ledger::open() 137 .map_err(|e| format!("opening the spend ledger: {e}")) 138 .and_then(|ledger| Self::new(key.trim(), binary, ledger, model, Endpoint::api())), 139 ) 140 }
Build a client against an already-open [Ledger]. Every real caller
wants [Self::from_env]; this is the constructor that makes the
dependency explicit rather than reached for internally, so a test can
hand it a ledger rooted in a private directory instead of the real
$XDG_STATE_HOME/jev - and so there is no Jev value anywhere that
was not required, at construction, to say where its spend is kept.
148 pub fn new( 149 key: &str, 150 binary: impl Into<String>, 151 ledger: Ledger, 152 model: ModelId, 153 endpoint: Endpoint, 154 ) -> Result<Self, String> { 155 // The trust anchors are compiled in rather than read from the host, so 156 // what this trusts is the same on every machine and in a sandbox with 157 // no /etc/ssl at all. The provider is RustCrypto: pure Rust, no ring, 158 // no aws-lc, nothing to cross-compile for the web build later. 159 let mut roots = rustls::RootCertStore::empty(); 160 roots.extend(webpki_roots::TLS_SERVER_ROOTS.iter().cloned()); 161 if let Some(path) = endpoint.ca_file() { 162 use rustls::pki_types::CertificateDer; 163 use rustls::pki_types::pem::PemObject; 164 let shown = path.display(); 165 for cert in CertificateDer::pem_file_iter(path).map_err(|e| format!("CA file {shown}: {e}"))? { 166 let cert = cert.map_err(|e| format!("CA file {shown}: {e}"))?; 167 roots.add(cert).map_err(|e| format!("CA file {shown}: {e}"))?; 168 } 169 } 170 let mut tls = rustls::ClientConfig::builder_with_provider(Arc::new(rustls_rustcrypto::provider())) 171 .with_safe_default_protocol_versions() 172 .map_err(|e| format!("rustls protocol versions: {e}"))? 173 .with_root_certificates(roots) 174 .with_no_client_auth(); 175 tls.alpn_protocols = vec![b"h2".to_vec()]; 176 let transport = HttpsTransport::new(endpoint, Arc::new(tls)); 177 let client = Client::new(transport, TokioRuntime, model, key).map_err(|e| e.to_string())?; 178 Ok(Self { 179 client, 180 binary: binary.into(), 181 ledger, 182 guards: Guards::from_env(), 183 last_rate_limit: Mutex::new(None), 184 }) 185 }
The same client under other guards than the environment's.
What the shared guards say right now - every process's questions, not only this one's.
The most recent [RateLimit] this client has seen, if any response
ever carried one.
Ask, under jev-client's retry policy - but first, have the shared
ledger admit it ([Ledger::admit]: question bucket, hour breaker, out
of credit - there is no lifetime cap, per the user's own ruling,
2026-09-22), before a single byte goes over the network. A refusal
that clears itself is [Failure::Throttled], one that does not (out
of credit) is [Failure::Budget]; neither is retried here.
why is recorded against the spend in the ledger's reason. It is a
required argument, not an option, so that every line of the ledger can
say why the money went: the breaker trip of 2026-09-20 had to be
reconstructed from token counts because no line said.
229 pub async fn ask(&self, state: &Json, questions: &Questions, why: &str) -> Result<Answered, Failure> { 230 let worst_case = worst_case_usd(self.client.model(), state, questions); 231 let hold = self.ledger.admit(ledger::now(), &self.guards, worst_case, &self.binary, why).map_err( 232 |refusal| match refusal { 233 Refusal::Throttled { why, retry_in } => Failure::Throttled { 234 why, 235 retry_in: Duration::try_from_secs_f64(retry_in).unwrap_or(Duration::MAX), 236 }, 237 other => Failure::Budget(other.to_string()), 238 }, 239 )?; 240 241 let started = Instant::now(); 242 match self.client.ask(state, questions).await { 243 Ok(answered) => { 244 let rate_limit = parse_rate_limit(&answered.headers); 245 if let Some(rate_limit) = rate_limit { 246 *self.last_rate_limit.lock().unwrap_or_else(PoisonError::into_inner) = Some(rate_limit); 247 if let Err(e) = self.ledger.note_rate_limit(&rate_limit) { 248 eprintln!("jev ledger: could not record a rate-limit snapshot: {e}"); 249 } 250 } 251 let usage = answered.response.usage(); 252 let cost = usage.dollars(); 253 let entry = Entry { 254 time: ledger::now(), 255 binary: self.binary.clone(), 256 input_tokens: usage.input_tokens, 257 output_tokens: usage.output_tokens, 258 cost_usd: cost, 259 request_id: answered.request_id.clone(), 260 estimated: false, 261 reason: Some(why.to_owned()), 262 hold: None, 263 }; 264 if let Err(e) = self.ledger.settle(&hold, entry) { 265 // The request already happened and was paid for 266 // whether or not this write succeeds; refusing to 267 // return the answer would not undo that, it would 268 // just also lose the decision. The hold stays 269 // unsettled, so the ledger goes on counting its 270 // worst case - more than it cost. Loudly logged, 271 // since a ledger that cannot be written to is a 272 // problem worth a human's attention on its own. 273 eprintln!("jev ledger: could not record a ${cost:.6} spend: {e}"); 274 } 275 Ok(Answered { 276 took: started.elapsed(), 277 attempts: answered.attempts, 278 body: String::from_utf8_lossy(&answered.body).into_owned(), 279 headers: collect_headers(&answered.headers), 280 request_id: answered.request_id, 281 rate_limit, 282 response: answered.response, 283 }) 284 } 285 Err(error) => { 286 if let ClientError::Api { error: api, .. } = &error 287 && api.kind == ApiErrorKind::OutOfCredit 288 && let Err(e) = self.ledger.mark_out_of_credit("the vendor reports no credit remaining") 289 { 290 eprintln!("jev ledger: could not record the vendor's out-of-credit answer: {e}"); 291 } 292 // `release` books the hold at nothing, which is true only 293 // when nothing can have been billed: the request was never 294 // built, or its one attempt was refused outright. After an 295 // interrupted or timed-out attempt the cost is unknown (the 296 // vendor reports usage only on success), so the hold stays 297 // and goes on counting its worst case, as a crashed 298 // process's does. 299 if nothing_billed(&error) 300 && let Err(e) = self.ledger.release(&hold, ledger::now(), &error.to_string()) 301 { 302 eprintln!("jev ledger: could not release an unanswered hold: {e}"); 303 } 304 Err(Failure::Client(error)) 305 } 306 } 307 } 308}
A conservative worst case, in tokens, for what one attempt could cost - used to check the guards BEFORE sending, when the true count (which only the response carries) is not yet known.
A UTF-8 character is never shorter than one token can be cheaper than, so counting characters can only overstate the token count - which is exactly what a worst case should do. The documented 64k-token ceiling per request caps it either way.
Every attempt jev-client may make (1 + retry::MAX_RETRIES) billed in
full.
Any of the conventional rate-limit header spellings, read opportunistically.
Digest §10: none of these are documented anywhere in the vendor's own
reference as of 2026-09-20, so this is expected to return None on every
real response; it stays general rather than named after one convention so
that if the vendor starts sending any of them, this starts reporting it
with no code changed anywhere that calls ask.
346fn parse_rate_limit(headers: &http::HeaderMap) -> Option<RateLimit> { 347 let text = |name: &str| headers.get(name)?.to_str().ok().map(str::to_owned); 348 let as_u64 = |name: &str| text(name)?.trim().parse::<u64>().ok(); 349 let as_f64 = |name: &str| text(name)?.trim().parse::<f64>().ok(); 350 let limit = as_u64("x-ratelimit-limit-requests") 351 .or_else(|| as_u64("x-ratelimit-limit")) 352 .or_else(|| as_u64("ratelimit-limit")); 353 let remaining = as_u64("x-ratelimit-remaining-requests") 354 .or_else(|| as_u64("x-ratelimit-remaining")) 355 .or_else(|| as_u64("ratelimit-remaining")); 356 let reset_seconds = as_f64("x-ratelimit-reset-requests") 357 .or_else(|| as_f64("x-ratelimit-reset")) 358 .or_else(|| as_f64("ratelimit-reset")); 359 (limit.is_some() || remaining.is_some() || reset_seconds.is_some()) 360 .then_some(RateLimit { limit, remaining, reset_seconds }) 361}
Every header name and value, redacting anything that could be a
credential by name - a bearer token echoed back, a cookie, anything with
"key"/"token"/"secret" in it. apps/jevprobe --headers is the one caller
that prints this; Answered always carries it so that mode needs no
separate request.
368fn collect_headers(headers: &http::HeaderMap) -> Vec<(String, String)> { 369 let sensitive = |name: &str| { 370 let lower = name.to_ascii_lowercase(); 371 ["authorization", "cookie", "token", "secret", "-key", "api-key"] 372 .iter() 373 .any(|word| lower.contains(word)) 374 }; 375 headers 376 .iter() 377 .map(|(name, value)| { 378 let name = name.as_str().to_owned(); 379 let value = if sensitive(&name) { 380 "«redacted»".to_owned() 381 } else { 382 value.to_str().unwrap_or("«non-utf8»").to_owned() 383 }; 384 (name, value) 385 }) 386 .collect() 387}
389#[cfg(test)] 390mod tests { 391 use super::*; 392 393 fn model() -> ModelId { 394 ModelId::pinned("jev-1.13.0").expect("a pinned model") 395 } 396 397 fn sandbox_ledger() -> Ledger { 398 let dir = std::env::temp_dir() 399 .join(format!("jev-http-test-{}", std::process::id())) 400 .join(ledger::now().to_string()); 401 Ledger::at(dir).expect("opening a sandbox ledger") 402 } 403 404 #[test] 405 fn a_missing_key_is_not_an_error() { 406 // Whatever this machine's environment happens to hold, an empty one 407 // means "no client", not "a broken client". 408 assert!( 409 Jev::new("", "test", sandbox_ledger(), model(), Endpoint::api()).is_ok(), 410 "an empty key still builds; the API refuses it" 411 ); 412 } 413 414 #[test] 415 fn rate_limit_headers_are_read_when_present_and_absent_otherwise() { 416 let empty = http::HeaderMap::new(); 417 assert_eq!(parse_rate_limit(&empty), None, "no header, no snapshot"); 418 419 let mut headers = http::HeaderMap::new(); 420 headers.insert("x-ratelimit-remaining-requests", "37".parse().expect("a header value")); 421 let rate_limit = parse_rate_limit(&headers).expect("a snapshot"); 422 assert_eq!(rate_limit.remaining, Some(37)); 423 assert_eq!(rate_limit.limit, None, "only what was actually sent is filled in"); 424 } 425 426 #[test] 427 fn a_bearer_token_echoed_back_would_be_redacted() { 428 let mut headers = http::HeaderMap::new(); 429 headers.insert("x-typesafe-request-id", "req_abc123".parse().expect("a header value")); 430 headers.insert("set-cookie", "session=deadbeef".parse().expect("a header value")); 431 let collected = collect_headers(&headers); 432 let value_of = |name: &str| { 433 collected.iter().find(|(n, _)| n == name).map(|(_, v)| v.clone()).unwrap() 434 }; 435 assert_eq!(value_of("x-typesafe-request-id"), "req_abc123"); 436 assert_eq!(value_of("set-cookie"), "«redacted»"); 437 } 438 439 #[test] 440 fn a_huge_state_is_capped_at_the_documented_ceiling_not_left_unbounded() { 441 let huge = Json::text(&"x".repeat(1_000_000)); 442 let mut questions = Questions::new(); 443 questions.noul("q", jev_protocol::Noul::new(Json::text("long?"))).expect("a question"); 444 assert_eq!(worst_case_tokens(&model(), &huge, &questions), 64_000.0); 445 } 446}