Transcript
[00:00] If you’re using AI in your actual application, you really want to pay attention to what TypeSafe has come out with. This brand new, what they’re calling a System 1 model called JEV. So they make this claim on their website that it’s 193.6 times faster and 444.6 times cheaper. And that’s not a lie, but what you’re getting is something very different than a large language model. And it’s actually something I think is going to be super, super useful for a lot of the applications I develop.
[00:32] So let’s talk about the cheaper claim. It is $42 per billion input tokens. You pay nothing for output tokens, zero. And if we compare that to something like Anthropics Claude, where you’re paying $1 to $10 per million input tokens and $5 to $50 per million output tokens, the difference is staggering.
[00:58] TypeSafe says JEV is not an LLM. It’s not a large language model. And so the question is right here on their website, is JEV just a smaller large language model like an SLM? And they say right here, it’s neither a small nor an LLM. And it’s off the Pareto curve. And if you want to know what the Pareto curve is. So what it’s doing is it’s optimized for a completely different task set. It’s not producing text or free-form text that you could use, say, in a chat bot. So JEV accepts text input only. It evaluates strings or JSON objects or arrays of strings. So it understands natural language input. But it does not produce replies or code or generate explanations of its reasoning. Instead, it returns type decisions and probabilities, which we can define with primitives. And there’s only three of these primitives. The first one is choice, where we can give it some input and tell it to select from a choice that we define. We can get a score. And we can define what that score range is. And we can give it a Newell, which is really just a Boolean. I don’t know what Newell means. I’ll have to look that up later.
[02:11] So I’ve got an API key for utilizing JEV. I’ve got a $5 balance that they gave me. This is not sponsored or anything like that. If you join the waitlist and they let you in, they give you $5 to try it out. So that’s what I’m going to do. I’ve got a directory here of a bunch of transcripts from videos I’ve created. So these aren’t small files. This is a lot of text. And what I’m going to have it do is hit the API and send it each transcript. So I’m going to go through all these. I’ve got about 10 there. And I’ve created three primitive questions. So the first one is a Newell, strong hook, does the opening of this transcript capture attention, blah, blah, blah. I don’t know if it can answer that. So I’m super curious to see how that goes. The second one is a choice. So which category best describes the content? And I give it some choices. This one, I’m pretty confident it can totally answer that for me. And then the third is a score, uh, uh, primitive. I, again, I’m not sure if it can actually answer this, but how likely is the topic to be actively searched low, medium or high? Uh, so the rest of this code is all just display stuff. I’m going to run this and we’re going to see what happens.
[03:25] And we’ll look at our usage and see how much of the $5 I used up and we’ll see what kind of responses we get here. So I’m going to run bun index TS everything should be wired up and, uh, it’s going to output a whole bunch of little, uh, terminal charts and stuff like that. Uh, so here we go. So each of these responses is coming back very quickly. Um, but I can see that on the retention hook, it has zero confidence so that it couldn’t answer that one. But as I suspected on the content category, it’s totally nailing that. And then the search potential, uh, it’s actually got confidence on those. So it’s not very high in that one. It’s 55% in this one, uh, 1.18 out of two. So that’s great.
[04:12] So let’s look at my usage and I’m at input of 21,269 tokens and output of 3,627. And it costs me one penny. And to be honest, I had run a smaller version of this, uh, a little bit before. So that’s in there as well. And technically it says less than a penny. So how is this useful for us?
[04:36] So how is this useful for us? Well, first the speed is ridiculous. And here we have, uh, folks at for cells saying that they were running this for their, uh, safety classifier, and it’s running five to 18 times faster and more accurate than GPT 5.6 Luna. And that jives with my own findings. The demo I just ran for you took, uh, uh, seconds, if that, and if I, if I take that and run it through, uh, like Muse or even Claude, we’re going to be on a scale of minutes. And that’s where this becomes really important because in your application where you’re evaluating information, as it comes in, we can use something like Jeff to make quick decisions. And then take those decisions and hand it off to a large language model if we need it.
[05:24] So for example, I have a software application that I run locally to manage my YouTube channel. And here I have a bunch of different feeds and information that I can grab from different websites and I’ll bookmark them. And then I do some research and I decide to turn those into what I call ideas. And then from there, I figure out whether or not I’m going to make a video about it. And one of the problems I’ve ran into is trying to get an LLM to, uh, deterministically figure out whether this is a good idea or bad idea based on a scoring system. I do have some of that in place, but it takes a long time. Integrating something like Jeff into the application is going to allow me to evaluate these ideas deterministically and come up with those answers before I hand it off to a large language model for further development, uh, almost free. I mean, certainly incredibly cheaply and almost every single example that typesafe has on their documentation site for Jeff is of this vein. So here are some example questions, what programming language is this in, what type of meeting was this, which, uh, product category is this. If we jumped to the score, how severe is the bug being reported? How formal is this outfit based on the description? These are all deterministic little tiny atomic questions that we can get answered quickly and cheaply without ever involving a large language model this early on in the process. These are all deterministic responses built on probabilities that we define. So we could use this to determine the nature of an email or which department a bug report belongs to, or break down the context of a user request. Uh, we could also use it to classify a prompt before deciding which LLM would be best to, uh, use that prompt with. And if you hop into their discord, you can see people are already creating all sorts of things with a model and a system that just came out yesterday. I think, uh, so it’s early days. It’s very exciting. Jev is super interesting.
[07:27] Go to the typesafe website, get on the wait list, and, uh, you’ll get at this moment, you’ll get $5 credit and you can start trying this thing out. I’m Joe Meadowlone, I make short, practical videos that respect your intelligence and your time. If that’s your style, you’ll like this channel.
JEV: The Tiny AI Model I Think Could Be Really Useful in Real Applications
If you’re using AI in an actual application, you really want to pay attention to what TypeSafe has come out with.
They’ve introduced what they’re calling a System 1 model called JEV.
TypeSafe makes a pretty wild claim on its website: JEV is 193.6× faster and 444.6× cheaper than the kinds of large language models we’re used to using.
And, based on what I just tried, I don’t think the interesting part is simply that it’s a cheaper model.
The interesting part is that JEV isn’t really trying to be an LLM at all.
JEV isn’t a smaller LLM
The first thing that caught my attention is that TypeSafe explicitly says JEV isn’t an LLM—and it isn’t even a small language model.
It’s optimized for a completely different task set.
JEV accepts text input. It can evaluate strings, JSON objects, and arrays of strings, and it understands natural-language input.
But it doesn’t generate a response like Claude or ChatGPT.
It doesn’t write code.
It doesn’t produce an explanation.
Instead, it returns type decisions and probabilities using a small set of primitives.
There are three of them:
- Choice — select from a set of choices you define.
- Score — return a score within a range you define.
- Boolean — answer a yes/no question.
That sounds much less impressive than an LLM until you start thinking about the kinds of decisions applications actually need to make.
So I tried it on my own data
I had an API key for JEV and a $5 credit balance TypeSafe provided. This isn’t sponsored; apparently if you join the waitlist and get access, they give you $5 to try it out.
So I decided to throw some real data at it.
I have a directory containing transcripts from videos I’ve created. These aren’t tiny files. There’s a lot of text in there.
I wrote a little Bun/TypeScript script that sends each transcript to JEV and asks three questions.
The first was a Boolean:
Does the opening of this transcript capture attention?
I wasn’t particularly confident that JEV would be able to answer something subjective like that.
The second was a choice:
Which category best describes this content?
For that one, I expected it to do pretty well.
The third was a score:
How likely is the topic to be actively searched?
Again, I wasn’t sure how useful the result would be.
Then I ran it.
The results were interesting
The responses came back very quickly.
On the retention-hook question, JEV had zero confidence for some of the results. So, at least in my little experiment, it wasn’t magically capable of answering every subjective question I threw at it.
But the content category was another story.
It was nailing those.
And the search-potential score was producing useful confidence values as well.
Then I checked the usage.
For the run, I had about 21,269 input tokens and 3,627 output tokens showing in the usage information.
The cost?
One penny.
Technically, the interface said it was less than a penny.
That’s pretty remarkable.
The speed is the other half of the story
The cost is obviously interesting, but I think the speed may be even more important.
The demo I just ran took seconds.
If I take the same kind of task and run it through a large language model, I’m potentially waiting on the scale of minutes when I’m processing a collection of these requests.
That’s a completely different proposition when you’re building an application.
You don’t necessarily need a giant model to make every decision.
Sometimes you just need to ask a tiny question:
- What category is this?
- Is this relevant?
- How severe is this?
- Which department does this belong to?
- Is this formal or informal?
- Which model should handle this request?
Those are small, atomic classification decisions.
And JEV seems to be designed specifically for them.
This is where I think it gets really interesting
I have a software application that I run locally to manage my YouTube channel.
It pulls in feeds and information from different websites. I bookmark things that look interesting, do some research, and eventually turn them into what I call ideas.
Then I have to decide whether an idea is actually worth turning into a video.
One problem I’ve run into is trying to get an LLM to deterministically decide whether something is a good idea or a bad idea based on a scoring system.
I have some of that in place already.
But it’s slow.
And that’s where I think something like JEV could fit extremely well.
Instead of handing everything to a large language model immediately, I can use JEV as an inexpensive first layer.
It can evaluate the incoming information, make a series of small decisions, and then I can take those decisions and hand the interesting stuff off to a much larger model for the expensive work.
The result is essentially an AI pipeline with a very cheap decision-making layer in front of the LLM.
And that’s a much more interesting architecture to me than simply replacing one LLM with another.
Think of JEV as a decision layer
The examples TypeSafe gives in its documentation are exactly the sort of thing I’m talking about.
Questions like:
- What programming language is this?
- What type of meeting was this?
- Which product category is this?
- How severe is the bug being reported?
- How formal is this outfit based on the description?
These aren’t requests for an essay.
They’re tiny decisions.
And those decisions can happen before you invoke a large language model.
That opens up a bunch of interesting possibilities.
You could use JEV to determine the nature of an email.
You could route a bug report to a department.
You could break down the context of a user request.
You could classify a prompt and decide which LLM should handle it.
You could evaluate incoming content and only send the things that pass your criteria to a more expensive model.
And because the output is based on probabilities and primitives that you define, the application can treat those results as structured data rather than trying to parse an LLM’s prose response.
It’s not replacing the LLM
I think this is the important distinction.
I’m not looking at JEV and thinking, “Great, I don’t need Claude or GPT anymore.”
That’s not what it’s for.
The interesting architecture is:
JEV → make a decision → LLM → do the complicated thing
Instead of:
LLM → figure out what to do → do the complicated thing
That first step can be extremely cheap and extremely fast.
And if you’re building an application that processes a lot of information, that matters.
There are still things I wouldn’t use it for
My little experiment also demonstrated an important limitation.
I asked it about the strength of a video hook.
It wasn’t particularly confident.
That makes sense.
A question like “does this opening capture attention?” is much fuzzier than “which category does this belong to?”
So I wouldn’t take this as evidence that JEV can replace a general-purpose model.
It’s a specialized tool.
And that’s exactly why I find it interesting.
It’s early
JEV had only just come out when I recorded this.
Even so, people were already building things with it in the TypeSafe Discord.
So it’s very early days.
I’m curious to see what people end up doing with a system that’s specifically designed around these tiny, fast, probabilistic decisions instead of generating arbitrary text.
Because that’s the part that I think has the most potential.
We spend a lot of time thinking about how to make LLMs more capable.
I’m increasingly interested in the opposite question:
What if some of the things we’re asking an LLM to do don’t need an LLM in the first place?
JEV is an interesting answer to that question.