I took a random book excerpt with 23,000 words (±30k input tokens) and used it as context. Jev still responds in 800ms. Other estimates put it between 50-100,000 tok/s prefill, which is obviously not possible with normal LLMs. Not even Cerebras is this fast.
There's some speculation that Jev is essentially an open weight model with novel post-training (RLCD). So, if these folks have competitive accuracy with just the base model, it could raise some questions about the necessity of Jev's architecture. You generally don't want to find yourself competing only on price.
Also, while it's clearly got a lot of training on some use cases, others that probably weren't in the training set have worse good decision rates than a random number generator. If you can rebuild the architecture, you can train it on your use case.
I mean just from what's known of the funding and timeline it pretty much has to be based on open weights.
But it is likely more than just a fine tune + novel training. At the very least the LM head is swapped out for a classifier one and then or also idk, bidirectional attention for the encoding pass I'm out of my depth at this point and will stop guessing. The training is probably where they have the biggest moat though, not that it's necessarily huge.
I have a project that fits jev as advertised almost comically well and I've been playing with it, and the various hacks and open versions. Jev doesn't necessarily perform better overall but it is quite different. It's sensitive to prompt phrasing in ways the others aren't, it's easy to generate questions where all the other models cluster in confidence but jev is an outlier. Not necessarily more correct, but it does feel like it's getting its answers in a different way.
I'm guessing just as much as anyone else but I've been spending a ton of time on this the last couple weeks, it landed right when I was most ready to dig into it.
Everyone is doing this to emulate Jev, but...
I took a random book excerpt with 23,000 words (±30k input tokens) and used it as context. Jev still responds in 800ms. Other estimates put it between 50-100,000 tok/s prefill, which is obviously not possible with normal LLMs. Not even Cerebras is this fast.
My question is why not use Jev instead? It's faster and cheaper.
There's some speculation that Jev is essentially an open weight model with novel post-training (RLCD). So, if these folks have competitive accuracy with just the base model, it could raise some questions about the necessity of Jev's architecture. You generally don't want to find yourself competing only on price.
Fyi, I haven't tested this yet.
Also, while it's clearly got a lot of training on some use cases, others that probably weren't in the training set have worse good decision rates than a random number generator. If you can rebuild the architecture, you can train it on your use case.
I mean just from what's known of the funding and timeline it pretty much has to be based on open weights.
But it is likely more than just a fine tune + novel training. At the very least the LM head is swapped out for a classifier one and then or also idk, bidirectional attention for the encoding pass I'm out of my depth at this point and will stop guessing. The training is probably where they have the biggest moat though, not that it's necessarily huge.
I have a project that fits jev as advertised almost comically well and I've been playing with it, and the various hacks and open versions. Jev doesn't necessarily perform better overall but it is quite different. It's sensitive to prompt phrasing in ways the others aren't, it's easy to generate questions where all the other models cluster in confidence but jev is an outlier. Not necessarily more correct, but it does feel like it's getting its answers in a different way.
I'm guessing just as much as anyone else but I've been spending a ton of time on this the last couple weeks, it landed right when I was most ready to dig into it.
These questions are answered by the OP (Same speed, image support) - additionally, GLM is open weight.