AI Decisions Just Got 100 Times Cheaper

Prompt engineering has always been rubbish. I wrote early last year about how it is the punch card era of programming all over again. If AI was so good at writing its own text, why could it not write the prompts as well?

Early in my use of large language models I came across DSPy, which solved a lot of the problems around my laziness with prompts.¹ It felt like an important step forward, and it felt like there was a useful product to be built on that idea. So I built my own version, Kaijo. It gave deterministic outputs and improved its own prompts: you gave it feedback, and it wrote the improvement for you. It even used its own system to create prompts for itself, which was both elegant and hard to get working. I ran a waitlist, gained some traction, used it on a few projects, and it was undeniably useful. However, something was always slightly missing.

Months later I worked out what that was. Optimising the prompts is only half the job. The AI also has to work out the structure of how it approaches the work. If I make the early decisions about how to break the job down, I am micromanaging it, and it never gets to find a better way.

Loops, Then Programs

My first step was to replace that structure with a powerful model and a loop, and let it work out the approach for itself. Loops are much simpler to run than any orchestrator, which is why I cut Ralph down to one line with no setup. This was a great start, and the job was normally done to a good standard. But even with the most powerful models there was some variability in the process, and letting the model call tools endlessly to work out what it needed cost a lot of tokens.

The next step is to codify that process. Ask the AI to write a structured, repeatable program which itself calls AI to get certain jobs done. Along the way you give the AI feedback where the structure does not quite work, and build up a set of repeatable cases, or evals, which it uses to check it has not gone off the rails while it makes improvements. I wrote up my latest thinking on this last week, when I explained why prompt evals alone are useless.

This breaks the use of AI into smaller pieces inside a larger program, which is a great way to build in the right amount of reliability. The repeatable parts matter as much as the AI parts. As I put it when I argued that you should stop saving tokens and start writing scripts:

The pattern matching is deterministic, so it lives in the script, where it runs in milliseconds and costs nothing. The judgement is not deterministic, so it stays in the prompt.

The Double Dovetail, from Stop Saving Tokens. Start Writing Scripts.

That is where some of the other Kaijo properties would be really useful. The self improvement not so much, as you need an agent to consider the whole build, but the single call simplicity, the guarantee that the output was well structured enough for a program to understand it, and the way you could choose a fast model for the job. Often you want a fast routing model at the beginning of the process, for example, as I talked about when I walked through building one for a client in my Make Your AI Rollout Stick webinar.

A Model That Decides but Cannot Talk

This week we got a new type of model with exactly those properties. TypeSafe calls it a System One model, trained with a process it calls Reinforcement Learning for Calibrated Decisions, and the first one is called Jev.² It can choose between options, give a score against some criteria, or answer yes or no as a probability. It brings nuance to a decision you might previously have hard coded, or paid an expensive LLM to make. It cannot generate text, but it can decide when you need text, so you call a traditional LLM only when you need one. It makes lots of decisions in parallel, it is fast, and it is very cheap: $42 per billion input tokens, with output tokens free.³ That is cheaper than DeepSeek 4.1 Flash on uncached input, with a guarantee that a machine can read every answer, and it answers all your questions at once.⁴

The guarantee is about the shape of the answer, not its truth. Jev cannot return an option that is not on your list, but it can still pick the wrong one, and the calibrated probability tells you how often that happens across many decisions, not which ones were wrong.⁵

High-res version available to newsletter subscribers

A Barrage of Questions

This is very powerful for systems that mix AI with regular code. We can hit any piece of input with every question that might be relevant, like asking for a whole multiple choice survey to be completed, and then let code act on the results. TypeSafe’s smart home demo shows the pattern. The code asks every question up front: is this a command or a question, which room, which device, which action. It throws away the answers that do not apply, and only hands over to a text generating model when someone needs a sentence back.⁶

Jev is behind a waitlist for now, but the approach is not exclusive to TypeSafe. Qwen’s open reranker models on Hugging Face already answer a yes or no question with a probability read straight from the model rather than generated as text, and more will follow.⁷

Where I Will Use It

In the text adventure I am building, a change in the world has to be checked against every subsystem that might care about it, and a decision model can work out which ones need updating and whether an LLM needs to run at all. For my email triage, I can ask a batch of questions about importance, urgency and safety, including who a message concerns, quickly and cheaply, and discard anything that is not relevant before an expensive model ever sees it. For batch jobs like the recipe importer or meal generator at Cherrypick, choosing between a few hundred ingredient types when reading a recipe, and deciding how complex it is to cook, is exactly this shape.

Decisions Just Got Cheaper

AI engineers need to learn it. Chief executives and their leadership teams need to know that simple AI decisions on large amounts of data potentially got hundreds of times cheaper overnight. That matters for any content filtering, triaging or routing your teams do, whether you thought it was too expensive to hand to AI, or whether people are currently doing it by pasting things into Claude.

Within six to twelve months, once more models are trained this way, this kind of decision model will be in every custom AI product, and you can bet today that the frontier labs are either scrambling to build their own or scrambling to buy TypeSafe.