First, congrats to the team on launching something genuinely interesting and new.
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.
I'm biased but I wouldn't call it misleading - generating text is super awesome and flexible, (we describe that in the blog post - and I personally use string models all the time) but it's true you pay a high tax for autoregressive generation
> Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do
From a quick look at this it looks like it could easily generate natural language text by following a structured representation like UMR (Uniform Meaning Representation) or the similar representation the Abstract-Wikipedia folks will be working on for generic encyclopedic text (which will be heavily informed by Universal Dependencies). These are basically linguistically principled and frame-based counterparts to a programming language AST, that can be then converted to natural language (in a broadly language-independent way, to the extent that semantics and pragmatics make that feasible) via some sort of NLG rendering.
His claim was that the title is misleading, not sure how it's relevant to that claim that you use "string models" (full LLMs).
The original title before it changed less than an hour ago was:
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm going to agree that was misleading.
And on the second point:
>>Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
>that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do"
Also going to disagree here, and I don't think it's semantics.
Just to be sure that I understand, you're saying that your model "can't hallucinate" because it only outputs a single thing, right? In this way, an LLM can't hallucinate either if I prompt it to do a classification task with a discrete set of possible outputs, right? (Assuming I reject non-conforming output. Actually, maybe what you're saying is that your system can't output non-conforming output?)
I don’t think it’s misleading if you compare on the use cases they suggested. It’s faster and cheaper (no idea if higher quality), so it’s immediately interesting for certain things.
And if you buy their RLCD claims, this might be even better than huge models that know a bunch of irrelevant things.
When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct
that's right, but because these models are probabilistic, it's also possible to be confidently wrong (and all future models will be smarter still and still have that possibility)
This sounds good but so far all claims just sound like marketing terms. I'd love to see real proof. e.g. "RLCD" and "parallel sampling" have nothing to back it up.
also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.
Nonetheless i want this to be true, so I'm looking forward to Jev
They have various benchmarks, e.g. how much time it takes them to do wikipedia page -> page games. Jev seems to take the same or fewer hops but in ~10x less time and for ~10x less money.
It's totally reasonable to compare against LLMs doing chain of thought if it gets comparable performance.
BTW it was not multi model playing doom, it was passing structured input and getting structured output. Its not what I thought: frames of video passed and real time game play.
It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing.
It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence).
Edit: On the AI primer page, it looks like they do the RLCD on a pre-trained base model?
I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable).
But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)
here is how I attempted to explain it to my company's AI group chat, is this roughly accurate?
"instead of autoregressive string output it instead outputs structured type-safe 'decisions' with probabilities/confidence scores, each generated in parallel
so sort of more like a Large Classification Model than a Large Language Model? or, maybe better to think of it as a sort of "shift left" in the LLM's transformer architecture, allowing you to replace the predefined token vocabulary of an LLM with a prescribed set of 'decisions' that need to be made based off the input context; and exposing those probabilities directly so they can be integrated into the system logic, instead of just sampling from top-K.
all of this while still being instruction-tuned (!!!)"
It's always been possible to build classification pipelines using LLM embeddings as the input. seems like this is a much more sophisticated / useful application of that concept
Hi - first congratulations, System One looks really promising.
The Doom demo really help me, at least, to understand how System One differs from LLMs. However the first demo (Side-by-side demonstration) - I'm struggling to understand what is going on here!
For many day-to-day computing use cases, Jev seems far better suited than an autoregressive language model, if for no other reason than it is not wasting compute thinking about anything other than how to spit out a decision.
Do you have an architectural explainer yet for Jev or are you holding that close to your chest and letting the magic rip for now?
I see this super interestingly as the "subconscious" to the llms "conscious" for lack of better terms. I'm super interested in this for broad and rapid decision making in the context of consumer agents so will be signing up for sure.
> the model takes as input a state (structured text? not sure if multi-modal)
Input, and criteria/instructions can both be defined as structured input (JSON). This ends up being pretty powerful because the model is trained to understand structure.
I assume this isn't really for consumers/individuals currently? Kinda feels like an improved magic 8 ball.
I can't really intuit how I should think about when the model will be accurate. Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
I would love for things like this to be accessible via hubs like open router or AWS bedrock. It's hard to justify adding new model vendors directly with all the heightened concerns about privacy and security, but if bold new capabilities are added to a centralized already-vendor like AWS, technical people can adopt them without going through a whole compliance/purchasing/vendor review process. And an extra middleman tax is well worth it when the cost savings of the model itself can be one-two orders of magnitude.
You can set privacy requirements and define an allow list. To me the main value prop is that I get one bill for all models and can quickly try new models without signing up anywhere or changing my code.
Oh! Also you can pass an array of models and if the first provider is down it automatically falls through to the next provider. More useful than it should be...
I was thinking about something similar (maybe) - generally speaking, embeddings for LLMs tend to learn real world concepts - things like 'fruit' or 'France' or 'city' as directions in embeddings.
But in things like programming, most concepts are abstract - 'if hungry eat an apple' in programming terms would look like
'if hunger > 50 {apples--; hunger-=30;}'
and compilers work with 'concept erasure' - to them, tokens (which are like llm tokens) look like
'if var1 > 50 {var2--;var1-=30}'.
They don't care about how these things map to real concepts. So all the embedding directions used to encode real-world concepts are just noise to LLMs when programming. This greatly reduces dimensionality and training costs. So does a token representation tuned for programming constructs, rather than natural language would probably have a more efficient encoding.
Current models go beyond the simple embedding because you start to encode groups of concepts in the context-aware part of the model (attention heads or any other method). So it is never simply words/tokens in isolation anymore.
I could see this being fantastic for classification tasks. Last year I shifted from using LLMs for bulk data classification tasks (1M transcripts) to generating embeddings and categorizing based on cosine similarity. It saved a ton of costs and time, but wasn't as accurate as LLMs. This seems like it can give me Terra-level classification ability with the cost/speed I need.
> Input tokens: $0.042 / MTok ($42 per billion tokens).
> Output tokens: FREE (too cheap to meter).
Insane. The video demos are really compelling, in particular the speed.
> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems.
I buy this vision. A lot of LLM integration I see these days is ultimately exactly this. OpenAI-style structured outputs works decently but this would be a great improvement in cost, latency.
constrained decoding (OpenAI-style structured outputs) make models dumber unfortunately - the short+dense version is that simply masking logits is insufficient because if ever a model was assigning probability to an invalid token, the model is by definition confused. you'd be better off erroring IMO
It could be used for coding if you gave it an AST.
If you work at TypeSafe please try this.
Side note: This is probably how LLMs would perform with better encoders and next-latent prediction, so eventually those will beat this architecture out. Still amazing though.
I've implemented tree-sitter in pi before, and while it works, I have no real proof it saves me tokens, or is more accurate. I think a better implementation is a model that's trained for AST's, not just "use tool, see what happens".
I'd love to do research on this when I have the time.
I saw the CEO reply elsewhere in the comments to some other question. Maybe he can shed some light on it. My gut feeling is that this is non-trivial and they did not get this to work (yet?), otherwise I can’t come up with a good reason as to why they would not demo that as I assume half of the crowd here (myself included) would line up as customers.
Yeah it would be quite trivial to try and implement an auto regressive AST generator for STLC with Jev provided that you had bounded variable names and integers.
As you said, if it worked, they would have demoed it haha
the hard part for coding is actually state engineering (e.g. getting your dependencies in context) - we haven't even tried it yet (because my philosophy is we should automate the easy tasks before the hard and we've been working on getting the model smart on the former)
we do think there's a lot of potential though and do want coding themed releases soon
I could see Jev being great at finding key symbols in codebase before a code generation/code review task. I sent you guys an email (to hello@) about using Jev in Code Review for www.ellipsis.dev.
We had early access and found it to be pretty useful. Having a second form of verification, where you can ask multiple questions (in the form of Nouls) raised our confidence in the outputs of other models. [0] IMHO This type of model works incredibly well in concert with LLMs, not as a replacement.
After much fumbling around with prompts and evals, this is exactly how I am using LLMs in production, to narrowly make choices and return structured data. Any deterministic work gets pulled out of the prompt and my goal is to narrow the model output to be as clearly defined and as minimal as possible.
Jev's focus on structured I/O and confidence scores are game changing. If this does at all what it claims, I think this is going to quickly become the new standard approach for agentic systems.
The doom video is also in the article itself (headline: "Doom").
I suppose this is the same video as the one from the parent comment, but I don't know for sure - I don't have a twitter account and the above link doesn't work for me.
But when their system is given the instruction "do not fire, simply dodge" - it doesn't "simply dodge", it actually gets close to the fleshy pink demon rather than keeping its distance. Or am I misunderstanding?
Okay, so it doesn't output text, that much is understood. What are the inputs like? I'm assuming maybe a text input? maybe an AST definition? Really hard to tell how this works at all from the demos, especially since we can't really try it out.
Congrats on the launch! What's different between Jev and Microsoft's Guidance package? https://github.com/guidance-ai/guidance Is it a diffusion generator under the hood?
Finetuning a language model for decision classification (with probabilities) is already well-understood. What specifically changes in the training objective with RLCD? Are its benefits isolated from Jev’s new architecture/parallelism?
Interesting concept, I can't see a reason to use a generalist classifier over an api rather then just training my own? If it was open weights I would probably mess around with it.
Will need hands on to truly tell, but the doom demo seems very promising. If it can play that with text descriptions of where stuff is by distance and degrees in a 3D context then many GUI automation tasks should be easily doable
I am positive I know exactly how this works, I made something similar a few months back. But the problem is without generation you are extremely limited in the use cases. And while the model can't hallucinate, it can still be wrong. It just can't make up data.
So an encoder-only model with a classifier trained on the heads or something? DeepSeek recently switched to an encoder-decoder architecture in an attempt to get the best of both worlds (fast prefill while preserving generation capability), I wonder if that might be the future?
That was the first thing that come into my head. OK I can train very simple model, that can generate json's for specific tasks, so what?
How we can be sure that this "limited use cases" not just overfitting for particular outputs (or even distillation?)
> We deliberately chose not to publish performance against public benchmarks. In fact, we plan to only have one-off evals when we make product updates.
lol, I bet they would publish them if their score on those benchmarks were good.
Signed up for the beta! :) would love to put this through some real-world shootouts against traditional LLMs to see where this type of model really excels.
I’m guessing it might be able to replace maybe 40-70% of LLM calls for a given pipeline depending on the business task, cutting the API costs on those calls by an order of magnitude.
Super intrigued by this - large scale automation using LLMs is quite annoying due to deprecation cycles of models from frontier labs and cost of running your own being prohibitive when you have a blend of them.
This is basically a zero-shot classifier that can accept raw text (or structured text) as an input, and is able to classify that text as accurately (they claim) as a frontier-level LLM. I have workflows this would be useful for, looking forward to it showing up on OpenRouter.
But also effectively this is a classification model. It excels at specific certain types of workloads, and obviously will fail at others. Not really sure how one benchmarks this tbf. I can see their argument on why this requires a novel specific eval for whatever your usecase is. A consistent "global" benchmark might be hard to do
Why did they pick the name System One? It's not really explained what "System One tasks" and "System One shaped queries" are. Things that need a fast response?
Does this imply it's a very small model? I couldn't find anything about the model itself.
If you zoom in (especially on the large title), you'll see that the text is a semi-transparent gray with a black internal outline. It seems like all the typography is SVG-rendered. Actually insane. I've never seen this before. Not even the most vibeslopped websites have that.
Parallel inference where you don't want a subagent seems niche. But there is a lot of random things where businesses ultimately want some kind of score instead of generating something.
I think the interesting thing would be seeing if prompt injections still work with this kind of model.
we have played with this! the fascinating thing we've found so far is that adversarial examples for our model are quite different from that of LLMs so that they work even better together
> LLMS
> Strings / generated text. Strings are flexible and can be anything: chat responses, code, hallucinations, refusals, or even type-safe structured values. To be used by software, responses need to be parsed + validated. There is also always some risk that the AI goes off the rails.
> Jev
> Type-safe structured values. Possible outputs and structure are defined in advance. The model never makes type errors. All answers are accompanied with calibrated probabilities and confidence scores.
I mean, this isn't even remotely comparable to LLMs so why compare? Also, why are they bringing up AGI given there approach is so restrictive that what they're building literally cannot have the creativity required for AGI? The video is 100% marketing slop...
The bulk of the application of LLMs is that they generate reasonably reliable text which doesn't need to be defined in advanced. I'm sure there is a niche for this and congrats to the team, but please let's not hype this as if it's the next big thing in AI...
It looks like a specialized encoder-only(-ish) transformer with scalar and ordinal output heads. Acausal in effect, maybe? Probably not even autoregressive?
I'd use this as a tool an LLM can use for specialized tasks. It's not AI in itself.
First, congrats to the team on launching something genuinely interesting and new.
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.
I'm biased but I wouldn't call it misleading - generating text is super awesome and flexible, (we describe that in the blog post - and I personally use string models all the time) but it's true you pay a high tax for autoregressive generation
> Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do
From a quick look at this it looks like it could easily generate natural language text by following a structured representation like UMR (Uniform Meaning Representation) or the similar representation the Abstract-Wikipedia folks will be working on for generic encyclopedic text (which will be heavily informed by Universal Dependencies). These are basically linguistically principled and frame-based counterparts to a programming language AST, that can be then converted to natural language (in a broadly language-independent way, to the extent that semantics and pragmatics make that feasible) via some sort of NLG rendering.
His claim was that the title is misleading, not sure how it's relevant to that claim that you use "string models" (full LLMs).
The original title before it changed less than an hour ago was:
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm going to agree that was misleading.
And on the second point:
>>Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
>that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do"
Also going to disagree here, and I don't think it's semantics.
Type safety is not factual correctness.
Just to be sure that I understand, you're saying that your model "can't hallucinate" because it only outputs a single thing, right? In this way, an LLM can't hallucinate either if I prompt it to do a classification task with a discrete set of possible outputs, right? (Assuming I reject non-conforming output. Actually, maybe what you're saying is that your system can't output non-conforming output?)
> I'm biased but I wouldn't call it misleading
- @CompleteSkeptic
Very strange.
I don’t think it’s misleading if you compare on the use cases they suggested. It’s faster and cheaper (no idea if higher quality), so it’s immediately interesting for certain things.
And if you buy their RLCD claims, this might be even better than huge models that know a bunch of irrelevant things.
When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct
Technically speaking when you send the prefix “The capital of France is “ into an LLM it will also produce probabilities across its whole vocabulary.
that's right, but because these models are probabilistic, it's also possible to be confidently wrong (and all future models will be smarter still and still have that possibility)
> Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring,
My first thought was that it would be ideal for robotics? As in control of limbs, general planning, route finding, etc.
This sounds good but so far all claims just sound like marketing terms. I'd love to see real proof. e.g. "RLCD" and "parallel sampling" have nothing to back it up.
also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.
Nonetheless i want this to be true, so I'm looking forward to Jev
Edit: I really have to say that I like their manifesto https://typesafe.ai/manifesto
They have various benchmarks, e.g. how much time it takes them to do wikipedia page -> page games. Jev seems to take the same or fewer hops but in ~10x less time and for ~10x less money.
It's totally reasonable to compare against LLMs doing chain of thought if it gets comparable performance.
love that you love the manifesto! letting the first batches off the waitlist now, but we do have some early users describing their experience (https://x.com/danshipper/status/2099947471518474522)
It's not an LLM though it's a frontier model on structured data
Did you see the video where it plays Doom, it made it click for me
BTW it was not multi model playing doom, it was passing structured input and getting structured output. Its not what I thought: frames of video passed and real time game play.
so what? put an LLM on Cerebras and get its responses faster, and put Jev on Cerebras and gets its responses even faster
A question I have, with the type { output: string }, would the model not become a LLM? And if it does, shouldn’t it cost the same as a LLM for output?
This is a good product but the naming/branding is pretty unfortunate.
Typesafe.AI sounds like some typescript/structured output type of tool…
What even is “system one” ?
IMO the product/tech is really there, just needs better communication.
I mean, it's a structured output model that (apparently) can't hallucinate. I don't mind the name.
It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing.
It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence).
Edit: On the AI primer page, it looks like they do the RLCD on a pre-trained base model?
[0]:https://docs.typesafe.ai/concepts/system-one
CEO here - that is right!
I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable).
But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)
here is how I attempted to explain it to my company's AI group chat, is this roughly accurate?
"instead of autoregressive string output it instead outputs structured type-safe 'decisions' with probabilities/confidence scores, each generated in parallel
so sort of more like a Large Classification Model than a Large Language Model? or, maybe better to think of it as a sort of "shift left" in the LLM's transformer architecture, allowing you to replace the predefined token vocabulary of an LLM with a prescribed set of 'decisions' that need to be made based off the input context; and exposing those probabilities directly so they can be integrated into the system logic, instead of just sampling from top-K.
all of this while still being instruction-tuned (!!!)"
It's always been possible to build classification pipelines using LLM embeddings as the input. seems like this is a much more sophisticated / useful application of that concept
Hi - first congratulations, System One looks really promising.
The Doom demo really help me, at least, to understand how System One differs from LLMs. However the first demo (Side-by-side demonstration) - I'm struggling to understand what is going on here!
For many day-to-day computing use cases, Jev seems far better suited than an autoregressive language model, if for no other reason than it is not wasting compute thinking about anything other than how to spit out a decision.
Do you have an architectural explainer yet for Jev or are you holding that close to your chest and letting the magic rip for now?
architecture is close to the chest for now, but we have talked about writing a paper
I don't want to shill my blog too much, but I will say data is probably far most interesting than architecture: https://www.completeskeptic.com/p/the-bitterest-lesson
I see this super interestingly as the "subconscious" to the llms "conscious" for lack of better terms. I'm super interested in this for broad and rapid decision making in the context of consumer agents so will be signing up for sure.
> the model takes as input a state (structured text? not sure if multi-modal)
Input, and criteria/instructions can both be defined as structured input (JSON). This ends up being pretty powerful because the model is trained to understand structure.
e.g.: https://docs.typesafe.ai/primitives/advanced#structured-inst...
> not sure if multi-modal
just JSON... for now :)
> outputs the question's answers as appropriate
correct!
I assume this isn't really for consumers/individuals currently? Kinda feels like an improved magic 8 ball.
I can't really intuit how I should think about when the model will be accurate. Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
Side note: it took me more time than I would like to admit to realize that Diogo Almeida isn’t a satirical version of the name Dario Amodei
That would have to default to Wario Amodei.
I feel like should be Cario Amodei. The D to C flip a rotation of the M to W flip
It wasn't until the demo videos that I realized the post wasn't satirical.
I would love for things like this to be accessible via hubs like open router or AWS bedrock. It's hard to justify adding new model vendors directly with all the heightened concerns about privacy and security, but if bold new capabilities are added to a centralized already-vendor like AWS, technical people can adopt them without going through a whole compliance/purchasing/vendor review process. And an extra middleman tax is well worth it when the cost savings of the model itself can be one-two orders of magnitude.
Isn't openrouter the exact opposite of caring about security and privacy?
I guess you can choose your provider still? But isn't the point that the lowest bidder is doing inference?
You can set privacy requirements and define an allow list. To me the main value prop is that I get one bill for all models and can quickly try new models without signing up anywhere or changing my code. Oh! Also you can pass an array of models and if the first provider is down it automatically falls through to the next provider. More useful than it should be...
I always setup guard rails so that only zdr providers are used.
The thing is, in this climate it's hard to believe such tech will remain secret for long.
So, assuming this is not vaporware, this would raise the tide for everyone because it shows what's possible.
I was thinking about something similar (maybe) - generally speaking, embeddings for LLMs tend to learn real world concepts - things like 'fruit' or 'France' or 'city' as directions in embeddings.
But in things like programming, most concepts are abstract - 'if hungry eat an apple' in programming terms would look like
'if hunger > 50 {apples--; hunger-=30;}'
and compilers work with 'concept erasure' - to them, tokens (which are like llm tokens) look like
'if var1 > 50 {var2--;var1-=30}'.
They don't care about how these things map to real concepts. So all the embedding directions used to encode real-world concepts are just noise to LLMs when programming. This greatly reduces dimensionality and training costs. So does a token representation tuned for programming constructs, rather than natural language would probably have a more efficient encoding.
Current models go beyond the simple embedding because you start to encode groups of concepts in the context-aware part of the model (attention heads or any other method). So it is never simply words/tokens in isolation anymore.
I could see this being fantastic for classification tasks. Last year I shifted from using LLMs for bulk data classification tasks (1M transcripts) to generating embeddings and categorizing based on cosine similarity. It saved a ton of costs and time, but wasn't as accurate as LLMs. This seems like it can give me Terra-level classification ability with the cost/speed I need.
yup just joined the waiting list with a very similar use case in mind
> Input tokens: $0.042 / MTok ($42 per billion tokens).
> Output tokens: FREE (too cheap to meter).
Insane. The video demos are really compelling, in particular the speed.
> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems.
I buy this vision. A lot of LLM integration I see these days is ultimately exactly this. OpenAI-style structured outputs works decently but this would be a great improvement in cost, latency.
thanks a ton!
constrained decoding (OpenAI-style structured outputs) make models dumber unfortunately - the short+dense version is that simply masking logits is insufficient because if ever a model was assigning probability to an invalid token, the model is by definition confused. you'd be better off erroring IMO
It could be used for coding if you gave it an AST.
If you work at TypeSafe please try this.
Side note: This is probably how LLMs would perform with better encoders and next-latent prediction, so eventually those will beat this architecture out. Still amazing though.
I've implemented tree-sitter in pi before, and while it works, I have no real proof it saves me tokens, or is more accurate. I think a better implementation is a model that's trained for AST's, not just "use tool, see what happens".
I'd love to do research on this when I have the time.
Cool project!
That's what I was insinuating through "better encoder"; the model creating more efficient representations of ASTs using something like JEPA
I saw the CEO reply elsewhere in the comments to some other question. Maybe he can shed some light on it. My gut feeling is that this is non-trivial and they did not get this to work (yet?), otherwise I can’t come up with a good reason as to why they would not demo that as I assume half of the crowd here (myself included) would line up as customers.
Yeah it would be quite trivial to try and implement an auto regressive AST generator for STLC with Jev provided that you had bounded variable names and integers.
As you said, if it worked, they would have demoed it haha
the hard part for coding is actually state engineering (e.g. getting your dependencies in context) - we haven't even tried it yet (because my philosophy is we should automate the easy tasks before the hard and we've been working on getting the model smart on the former)
we do think there's a lot of potential though and do want coding themed releases soon
I could see Jev being great at finding key symbols in codebase before a code generation/code review task. I sent you guys an email (to hello@) about using Jev in Code Review for www.ellipsis.dev.
We had early access and found it to be pretty useful. Having a second form of verification, where you can ask multiple questions (in the form of Nouls) raised our confidence in the outputs of other models. [0] IMHO This type of model works incredibly well in concert with LLMs, not as a replacement.
[0] https://goodstartlabs.com/research/verification-is-the-bottl...
After much fumbling around with prompts and evals, this is exactly how I am using LLMs in production, to narrowly make choices and return structured data. Any deterministic work gets pulled out of the prompt and my goal is to narrow the model output to be as clearly defined and as minimal as possible.
Jev's focus on structured I/O and confidence scores are game changing. If this does at all what it claims, I think this is going to quickly become the new standard approach for agentic systems.
we hope so! the bigger hope is to not just eat LLM market share, but to allow for people to use AI much more in the inner loop of software
https://x.com/completeskeptic/status/2099925682726002904?s=4...
The doom demo is quite cool
The doom demo is also in the article, for anyone that doesn't want to go to X.com. :)
Direct link to the Doom video tweet:
https://x.com/completeskeptic/status/2099925687465570372
The doom video is also in the article itself (headline: "Doom").
I suppose this is the same video as the one from the parent comment, but I don't know for sure - I don't have a twitter account and the above link doesn't work for me.
I linked to the tweet that has the video because if you are not signed in you cannot see the whole thread of tweets.
I can see the individual tweets in the browser while not signed in though.
But when their system is given the instruction "do not fire, simply dodge" - it doesn't "simply dodge", it actually gets close to the fleshy pink demon rather than keeping its distance. Or am I misunderstanding?
i think its just telling the model that it cant output a shoot action
Link to a raw MP4 of the video, from the parent article: https://framerusercontent.com/assets/rlL7ImEbISFoYt3IJEHHfvj...
It's in the parent article under a section named "Doom" in case that asset URL ever changes.
So in theory you could feed it incomplete text, and then ask it for the probabilities of what the next character could be?
I think the joke here is getting missed
you could, but it the model is not optimized for text
this is complex, but generating text is highly complicated and requires mode dropping to make long cohesive text
If you provide it an AST of the english language, yes.
Okay, so it doesn't output text, that much is understood. What are the inputs like? I'm assuming maybe a text input? maybe an AST definition? Really hard to tell how this works at all from the demos, especially since we can't really try it out.
From the person in the video regarding issues with benchmarks in general, and for LLMs. Also their approach. Good article.
https://substack.com/home/post/p-215252866
They never show exactly how they use it? Only a bunch of animations of it 'working'. Would like to see the actual code used for the demos!
The doom demo shows the program state / query.
> [others] Output tokens: ~5x more expensive than input tokens.
> [them] Output tokens: FREE (too cheap to meter).
I'm very confused by this.
They're not doing autoregression, so all the outputs are computed in one big forward pass. Very cheap.
I think OP is confused about "others" vs "them".
they’re talking about two totally different things, right?
it's our output tokens that are free (under the system one / jev column)
Congrats on the launch! What's different between Jev and Microsoft's Guidance package? https://github.com/guidance-ai/guidance Is it a diffusion generator under the hood?
"While Jev gives up string generation, it’s optimized for structured outputs and can’t hallucinate"
Ouh! Any open weights models that can do this yet?? If not, how much longer? I have a Mac Studio coming soon.
If I’m understanding correctly, this will work well for self driving cars?
Congrats on the release!
Finetuning a language model for decision classification (with probabilities) is already well-understood. What specifically changes in the training objective with RLCD? Are its benefits isolated from Jev’s new architecture/parallelism?
Interesting concept, I can't see a reason to use a generalist classifier over an api rather then just training my own? If it was open weights I would probably mess around with it.
Will need hands on to truly tell, but the doom demo seems very promising. If it can play that with text descriptions of where stuff is by distance and degrees in a 3D context then many GUI automation tasks should be easily doable
So is it a structured data-based language model? Or is there a model and a harness? Hopefully they’ll open up and explain more.
it is just a model, no harness yet ;)
it is a structured data model, but technically not a language model (it doesn't generate language)
I am positive I know exactly how this works, I made something similar a few months back. But the problem is without generation you are extremely limited in the use cases. And while the model can't hallucinate, it can still be wrong. It just can't make up data.
Are you able to share how it works in that case?
I'll tell you this. Output isn't too cheap to meter, there is no decoder.
So an encoder-only model with a classifier trained on the heads or something? DeepSeek recently switched to an encoder-decoder architecture in an attempt to get the best of both worlds (fast prefill while preserving generation capability), I wonder if that might be the future?
That was the first thing that come into my head. OK I can train very simple model, that can generate json's for specific tasks, so what? How we can be sure that this "limited use cases" not just overfitting for particular outputs (or even distillation?)
Except this, this thing looks like revolution.
> We deliberately chose not to publish performance against public benchmarks. In fact, we plan to only have one-off evals when we make product updates.
lol, I bet they would publish them if their score on those benchmarks were good.
What's the difference compared to just taking an embedding and feed forward a simple net trained for the task?
Is there a downloadable technical report somewhere?
This puts the human even more out of the loop I'll guess?
I’m not understanding what this is. It’s a faster cheaper LLM?
Signed up for the beta! :) would love to put this through some real-world shootouts against traditional LLMs to see where this type of model really excels.
I’m guessing it might be able to replace maybe 40-70% of LLM calls for a given pipeline depending on the business task, cutting the API costs on those calls by an order of magnitude.
Funny how it can do everything but not chat. Sort of how when I was a kid I thought of a medicine that could cure any disease except the common cold.
Is the tradeoff of the parallel output that we don't get arbitrary string generation? like output # of tokens is fixed ahead of time?
Either way, really cool and impressive.
Super intrigued by this - large scale automation using LLMs is quite annoying due to deprecation cycles of models from frontier labs and cost of running your own being prohibitive when you have a blend of them.
Wild that it doesn't generate text. I wonder how its technology compares to Tesla's FSD stack.
So... a classifier model?
This is basically a zero-shot classifier that can accept raw text (or structured text) as an input, and is able to classify that text as accurately (they claim) as a frontier-level LLM. I have workflows this would be useful for, looking forward to it showing up on OpenRouter.
exactly right!
This sort of stuff almost sends shivers down my spine, it's like i'm looking 5 years into the future.
There was no "AI Winter"
Can this be used in practice to write code?
> Extraordinary claims require extraordinary evidence so see below for the receipts.
Yes, that’s the kind of attitude I want to see in these model releases
But the evidence is not there...
Indeed, they talk as skeptics but don’t offer a ton of evidence, other than a couple videos of demos. A live demo would be far more convincing.
They gesture at not using benchmarks for some reason...
https://typesafe.ai/blog/antibenchmaxxing
But also effectively this is a classification model. It excels at specific certain types of workloads, and obviously will fail at others. Not really sure how one benchmarks this tbf. I can see their argument on why this requires a novel specific eval for whatever your usecase is. A consistent "global" benchmark might be hard to do
Why did they pick the name System One? It's not really explained what "System One tasks" and "System One shaped queries" are. Things that need a fast response?
Does this imply it's a very small model? I couldn't find anything about the model itself.
Maybe: https://thedecisionlab.com/reference-guide/philosophy/system...
Bingo. It's a Psychology term for the part of our brain that reacts instinctively rather than thoughtfully and logically
Woof, that page is hard to read. I don't understand what they've done to the way text is rendering but it's not great for my eyes.
If you zoom in (especially on the large title), you'll see that the text is a semi-transparent gray with a black internal outline. It seems like all the typography is SVG-rendered. Actually insane. I've never seen this before. Not even the most vibeslopped websites have that.
I could put this to use today.
I think we'll see a bunch of different architectures over the next five years.
Parallel inference where you don't want a subagent seems niche. But there is a lot of random things where businesses ultimately want some kind of score instead of generating something.
I think the interesting thing would be seeing if prompt injections still work with this kind of model.
we have played with this! the fascinating thing we've found so far is that adversarial examples for our model are quite different from that of LLMs so that they work even better together
um what is going on with the outfit changes in the launch video...
https://x.com/CompleteSkeptic/status/2099925682726002904
Can't tell if they're just having fun or if it is ai-generated. On the verge of not being able to tell. Voice sounds a little synthetic.
definitely not AI-generated - this is my real wardrobe
we also thought the voice at the end was AI-ish, but apparently that's a real voice actor but slightly sped up
The whole video seemed generated to me
Pretty sure that's an intended joke.
Reminds me of this: https://www.reddit.com/r/ITcrowd/comments/tg05j1/i_cant_beli...
What is it about the rendering of this page that is so... off? It almost looks like the entire thing is a <canvas> element.
edit: looks like a framer export where there is a text stroke being applied :|
Looks like a great model for NLP.
this is interesting, so not an LLM but can be used in these use cases that LLM's have been shoehorned into
https://docs.typesafe.ai/concepts/use-case-map
oooooh it can play Doom!
forget LLM benchmaxxing sidequests, I'm sold on the real benchmark
"is this the real thing or is just fantasy"
If this is true means, AI Stock bubble burst. (For good)
> Outputs
> LLMS > Strings / generated text. Strings are flexible and can be anything: chat responses, code, hallucinations, refusals, or even type-safe structured values. To be used by software, responses need to be parsed + validated. There is also always some risk that the AI goes off the rails.
> Jev > Type-safe structured values. Possible outputs and structure are defined in advance. The model never makes type errors. All answers are accompanied with calibrated probabilities and confidence scores.
I mean, this isn't even remotely comparable to LLMs so why compare? Also, why are they bringing up AGI given there approach is so restrictive that what they're building literally cannot have the creativity required for AGI? The video is 100% marketing slop...
The bulk of the application of LLMs is that they generate reasonably reliable text which doesn't need to be defined in advanced. I'm sure there is a niche for this and congrats to the team, but please let's not hype this as if it's the next big thing in AI...
It looks like a specialized encoder-only(-ish) transformer with scalar and ordinal output heads. Acausal in effect, maybe? Probably not even autoregressive?
I'd use this as a tool an LLM can use for specialized tasks. It's not AI in itself.
If this is true, then AI Stock Bubble burst (for Good)