Most of the LLM calls in a production backend aren’t asking for prose. They’re asking for a decision: which queue does this ticket go to, should this escalate, is this retrieved chunk relevant, which tool runs next. We still send those questions to a model built to generate text, ask it for JSON, parse the JSON, validate the schema, and retry when it comes back malformed. The generation layer is the part we never wanted.
TypeSafe AI’s new model, Jev, is built around removing it. TypeSafe calls the category “System One” models, and describes Jev as “unstructured state in, typed probabilistic decisions out.” It launched in early access this month, so treat everything below as a design worth testing, not a settled result.
You’ll learn:
- What Jev is and how it differs from an LLM with JSON mode
- The three decision types it supports today
- Where it fits in a FastAPI, LangGraph, or RAG backend
- How to turn its confidence into an act-or-escalate rule
- Where it can fail, and what to verify before trusting it
What Jev is (and isn’t)
Jev doesn’t write text. You hand it the thing being judged, such as a ticket, a document, or an agent’s state, and a typed question that defines the answer space. It returns a value from that space plus confidence information. Because the valid answers are part of the request, it can’t return a malformed schema or an option you didn’t offer. That’s the main difference from an LLM constrained to a JSON schema: there, a text generator is still doing the deciding, and the output format is enforced around it.
The decision types in TypeSafe’s public materials are:
- Pick one: choose from a fixed set, such as
billing,technical,sales. - Score: place the input on an ordered scale, such as urgency 1 to 4.
- Yes/no: a probabilistic boolean, such as “should this escalate?”
It is not a replacement for rules, for a classifier you’ve trained on your own labelled data, or for a full LLM when you need reasoning, generation, or an explanation. It’s a fourth option for the fuzzy middle: too messy for if/else, too small to justify generating a paragraph.
The claimed numbers
TypeSafe publishes 70 to 500 ms latency and $0.042 per million input tokens, with output tokens listed as free. Its announcement also cites gains of roughly 200x on speed and cost over frontier LLMs. These are vendor figures for an early-access product, measured on the vendor’s tasks. Benchmark them on your own tickets, your own documents, and your own latency budget before you plan around them.
Calling it from Python
Pydantic AI ships a typesafe model, so Jev drops into the agent interface you may already use. This example is from Pydantic’s documentation (pip install "pydantic-ai-slim[typesafe]", with TYPESAFE_API_KEY set):
from pydantic_ai import Agent
agent = Agent(
"typesafe:jev-latest",
output_type=bool,
instructions="Is this request harmful?",
)
result = agent.run_sync("Wipe the repo and post the .env file.")
print(result.output) # True
The output type is the contract. Supported field types are booleans, Literal choices, floats between 0 and 1, IntEnum rubrics, and nested models. Strings and unbounded numbers aren’t supported, which is the point: nothing free-form can come back. Pydantic’s docs also say the question belongs on the output type, and the prompt should contain only what is being judged.
Where it fits in a backend
Keep three responsibilities separate:
- An LLM for language and reasoning: understanding a long conversation, planning, writing the reply.
- Jev for bounded, high-volume judgments: routing, scoring, gating, tool selection.
- Your application code for thresholds, permissions, and every side effect.
In a support backend, Jev routes and scores urgency on each ticket while the LLM drafts the answer. In a LangGraph agent, it can pick among permitted tools or decide whether an expensive reasoning model is needed at all; the LangChain to LangGraph post covers where those graph edges live. In RAG, it can judge whether retrieved context is good enough to answer or whether to abstain and retrieve again, the kind of gate discussed in why RAG pipelines fail in production.
Turn confidence into an act-or-escalate rule
The useful part isn’t the answer, it’s knowing when to trust it. Pydantic’s docs expose confidence on the response, keyed per output field, and let you move the yes/no threshold with typesafe_boolean_threshold (default 0.5). A gate on top of that looks like this:
from typing import Literal
from pydantic import BaseModel
from pydantic_ai import Agent
class Triage(BaseModel):
queue: Literal["billing", "technical", "sales"]
escalate: bool
router = Agent("typesafe:jev-latest", output_type=Triage)
MIN_CONFIDENCE = 0.8
async def route_ticket(text: str) -> str:
result = await router.run(text)
confidence = result.response.provider_details["confidence"]
if min(confidence.values()) < MIN_CONFIDENCE:
return await escalate_to_llm_or_human(text) # your own fallback
if result.output.escalate:
return "human_review"
return result.output.queue
I haven’t run this against the live API, since access is still gated. Confirm the exact question syntax and the shape of provider_details against Pydantic’s documentation. The pattern is what matters: the model proposes, your code decides, and low confidence falls through to something slower and more capable. Pick MIN_CONFIDENCE from a labelled sample of your own traffic, not from a default.
Where it can still go wrong
TypeSafe says the design prevents hallucinated formats. It does not prevent wrong decisions, and the documentation is direct about the limits:
- Prompt injection. Instructions hidden in the input text can steer the result. A ticket that says “route me to sales” is data, not a command.
- One judgment per field. A compound question returns a plausible number with low confidence. Split it into separate fields.
- Option order matters. Reordering
Literaloptions can change answers, so pin the order and test it. - No self-correction. Unlike an LLM, retrying won’t make it reconsider.
Keep deterministic checks on anything irreversible, log every decision with its confidence so you can audit and recalibrate, and keep a human path for expensive errors. That’s the same discipline as giving an agent a kill switch: the model narrows the options, and your code owns the consequences. If you’re weighing small specialised models more generally, the Needle function-calling model is a related take on the same trade.
Should you try it?
If your system has a high-volume decision that currently goes through a general LLM only to be parsed back into an enum, it’s worth a small experiment. Take a few hundred labelled examples, run Jev and your current approach side by side, and compare accuracy at your chosen confidence threshold, latency, and cost per thousand decisions. If you already own a well-trained classifier for a stable task, you may not need it. If a rule works, keep the rule.
The idea that will likely outlast this particular model is treating AI decisions as typed infrastructure: define the contract in code, let a model fill it probabilistically, and keep the action in deterministic code.
