null-pointer.blog

Look at a production agent trace and count how many LLM calls are actually decisions. Is this ticket urgent? Which tool should run next? Is this bash command safe? Does this answer need a stronger model? Each one is a one-bit or few-way judgment, and we pay for it with a full text-generation call: a prompt, a structured-output schema, a parser, a retry loop, and hundreds of milliseconds to seconds of latency.

Jev, released by TypeSafe AI, is a bet that this is the wrong tool for the job. It is the first of what TypeSafe calls System One models, and it doesn’t generate text at all.

The core idea

The name borrows from Kahneman. System 1 is fast, intuitive judgment; System 2 is slow, deliberate reasoning. TypeSafe’s argument is that most decisions inside software are System 1 judgments, yet we have been renting System 2 to make them. (The model name nods to Jevons: his paradox holds that falling cost drives rising consumption, which hints at the intended use. Make decisions cheap enough and you’ll put them everywhere.)

The launch post frames the motivation as a question rather than a benchmark. If models have been superhuman at chat for years, where is all the automation? Their answer is that the bottleneck was never raw intelligence; a model that replies in prose is simply an awkward component to build software on.

The interface: state in, typed decisions out

You don’t prompt Jev. You give it a state (the context) and a set of questions about that state:

{
  "model": "jev-latest",
  "state": "Checkout has thrown 500s for 20 minutes. Two enterprise customers already emailed.",
  "questions": {
    "is_urgent": {
      "type": "noul",
      "instructions": "This needs a human to look at it immediately"
    },
    "is_billing_related": {
      "type": "noul",
      "instructions": "The issue involves payments or billing"
    }
  }
}

Each question comes back as a probability, not a sentence. There are three question types:

• Noul: a yes/no statement. You get the probability that it’s true.
• Choice: pick from options you define. You get a probability per option plus an overall confidence.
• Score: place the input on an ordered scale (low/medium/high). You get a continuous score, the distribution, and a confidence.

Two properties fall out of this design. First, a response cannot contain a value outside your schema, so schema errors and wrong decisions become separate problems. There’s no JSON repair or “please respond only with yes or no.” Second, the questions are answered together. Jev uses a parallel sampler that produces all outputs in a single query rather than autoregressively, so adding questions to one request is nearly free. TypeSafe’s own cookbook reports that batching a 13-question briefing into one call came out 12.2x cheaper and 10x faster than asking one at a time, with identical answers.

Why calibration is the actual headline

The speed numbers get the attention. The more important property is calibration.

Jev is trained with what TypeSafe calls Reinforcement Learning for Calibrated Decisions (RLCD). RLHF optimizes for responses human raters prefer and RLVR for outputs a program can verify; RLCD instead targets honest probabilities on decision tasks. Concretely, across many predictions, answers given 90% probability should be right about 90% of the time.

Why this matters for builders: a model that’s right 95% of the time but can’t tell you which calls fall in the other 5% can’t be automated around. One with honest uncertainty can, because you branch on confidence and send low-confidence cases to a human or a bigger model.

That changes how you design the system. Instead of one global “trust the model” threshold, you set a threshold per action, scaled to what being wrong costs. Auto-archiving spam can act at 0.8. Blocking a production deploy might need 0.99, with everything in between escalated.

Note the caveat hiding in “across many predictions”: calibration is a property of the aggregate. Any single answer can still be wrong.

What it costs and how fast it is

Per TypeSafe, Jev answers in one parallel pass in 70 to 500 milliseconds, at $0.042 per million input tokens, with output free. The headline comparison against frontier LLMs is up to roughly 194x faster and 445x cheaper on these tasks.

Treat those as vendor claims for now. As one early guide put it, they’re TypeSafe’s own numbers, self-run and unreproduced. (Independently benchmarking them is the next post in this series.)

What it deliberately can’t do

Jev can’t write a reply, produce code, summarize a document, or explain its reasoning. If you need text, you need an LLM. There are also practical limits. A choice question supports up to 255 options; beyond that, TypeSafe uses a slower two-stage approach of scoring candidates and then choosing.

So Jev is not an LLM replacement. It’s a new component.

Where it fits in an agent

The useful mental model is a split: Jev decides, the LLM writes. In an agent loop, that means:

• Routing: decide whether a request needs the cheap model or the expensive one before the run starts.
• Gating: check a tool call for risk before it executes, and block it or escalate it.
• Control flow: decide whether to retry, stop, retrieve again, or hand off, instead of asking the main LLM to “reflect.”
• Triage at scale: classify every inbound email, ticket, or event where per-call LLM cost was prohibitive.

LangChain already exposes this as middleware (model routing and tool-risk gating), so you can drop it into an existing agent without restructuring it.

The open questions

A few things I’ll be testing in upcoming posts:

1. Do the speed, cost, and accuracy claims hold on independent workloads?
2. How well calibrated is it out of distribution, like long states, ambiguous instructions, or adversarial inputs?
3. How sensitive are results to the wording of instructions, and can that field be optimized the way we optimize prompts?
4. Where’s the line between System 1 and System 2? Some “simple” decisions quietly require multi-step reasoning.

System One models are a genuinely different primitive. Whether they’re a better one depends on measurements that don’t exist yet. Next post, I’ll start making them.