Jev Limitations: What It Gets Wrong and How to Work Around It
Last checked · Independent guide, not affiliated with TypeSafe AI
Jev 1.13 is fast and well calibrated on simple judgments, but it reads questions very literally and is unreliable at arithmetic, counting, comparing dates, multi-hop reasoning and long states full of irrelevant text. It also cannot generate text, reads only English well, and accepts text only. Most workarounds come down to doing the logic in code and asking Jev narrow questions.
TypeSafe publishes an unusually frank list of the “jagged edges” in Jev 1.13, last reviewed September 17, 2026. This page summarizes it in our own words, adds the input limits from the model documentation, and includes one example from our own testing.
At a glance
Section titled “At a glance”| Weak spot | What goes wrong | Workaround |
|---|---|---|
| Literal reading | Answers the question as written, not as meant | Spell out the exact condition; put boundary cases in criteria |
| Math and numbers | Unreliable arithmetic and numeric comparison | Compute in code; give Jev the result or a named bucket |
| Counting | Miscounts items, words or occurrences | Ask one question per item and add up in code |
| Dates and times | Cannot reliably order dates or measure gaps | Extract date parts with Choices; compare in code |
| Indirection | Double negatives and multi-hop questions lose accuracy | Ask directly; point to the relevant field by name |
| Large, noisy state | Accuracy drops as unrelated text grows | Filter and send only what the question needs |
| Adversarial content | Text inside the state can steer the answer | Precise criteria; test injection cases; gate risky actions |
| Contradictory setup | Instructions and criteria that disagree confuse it | Keep criteria consistent with the question |
| Structural consistency | Related questions do not obey logical identities | Ask each decision one way; enforce logic in code |
| Generation | Cannot write text | Use an LLM for text; let Jev choose among candidates |
The weak spots in more detail
Section titled “The weak spots in more detail”Literal reading. Jev answers the words you wrote. In our own test, “Is the customer asking for money back?” scored 0.50 on a message that complained about a charge but never mentioned a refund, while “Does the customer say a charge might be wrong?” scored 0.97. Neither is wrong; the first question was ambiguous. When you catch yourself explaining what you really meant, add that explanation to the instructions or criteria. See Noul.
Math, counting and numeric formats. TypeSafe is blunt that Jev is not a calculator. It recognizes the shape of an answer rather than tallying, so counts go wrong more as lists get longer. It also does worse on raw numeric representations, such as hex color codes, than on names for the same thing. Do arithmetic in code, and when a question depends on a number, pass the computed value or a label such as “over budget”.
Score values are not measurements. A Score answer can land between levels (1.6 on a 0 to 2 scale). Use it for thresholds and ranking, not to recover a precise quantity: TypeSafe warns that interpolating between levels is poorly calibrated.
Dates. Jev reads dates as text. Asking which of two dates is earlier, or whether a date falls within a window, is unreliable, especially with mixed formats or relative references like “next Tuesday”. Extract the day, month and year as separate Choice questions (with a “not stated” option) and do the date math in code.
Indirection. Questions about a property of a property, or with double negatives, cost accuracy. Name the part of the state you mean, for example with a path like ticket.messages[0].text, and ask one hop at a time. If a later question depends on an earlier answer, make a second request.
Long, noisy states. Jev suffers from what TypeSafe calls context rot: accuracy falls as the state fills with material unrelated to the question, and debugging gets harder. Retrieve and filter first. The hard limit is about 32,000 tokens for the state plus the longest question; see max_tokens_exceeded.
Adversarial content. Jev does not treat the state as hostile by default. An injected instruction or a message that argues for its own classification can move the answer. Test prompt-injection cases before trusting Jev as a security gate, and do not let a single answer trigger irreversible actions.
Structural consistency. Asking the same thing as a Noul and as a yes/no Choice gives numbers that are not directly comparable, and a question and its negation will not always sum to 1. Pick one formulation per decision and compute derived logic in code.
Generation. Jev cannot write text. You can force it to build strings by chaining choices, but TypeSafe says this is slow and works poorly. Let a regex or an LLM propose candidates, then have Jev choose.
Input limits
Section titled “Input limits”- Text only. Strings, JSON objects and arrays. No images, audio, video or binary files. Convert them to text first.
- Language. English is Jev’s primary training language and where it is most accurate. Other languages, including Chinese, Japanese and Korean, are accepted but less accurate. Test on your own content and watch confidence closely.
- Size. About 32,000 tokens for the state and longest question; 64,000 per request including all questions.
- No fine-tuning. Everyone uses the same weights. You adapt Jev through the state, instructions and criteria. See Can you fine-tune Jev?
Reading the list the right way
Section titled “Reading the list the right way”None of these limits is unusual for a model that answers in a few hundred milliseconds. The design assumption is that code does the reasoning, math and bookkeeping, and Jev supplies the judgment that needs language understanding. When a task needs slow, multi-step thinking, use a reasoning model for that step. See Jev vs LLMs.