We found an approach to get Jev-like properties from standard LLMs like GLM-5.3-Flash.
The core idea is to craft the input prompt so that the first output token answers the question. This makes it possible to get a decision with a single forward pass.
In the blog post, we describe the approach in detail for GLM-5.3-Flash and vLLM. We benchmark this setup against Jev and Laya. We find that our setup is on-par with Jev in terms of accuracy and speed and that it substantially outperforms Laya.
Still, in terms of costs per decision, Jev is several x better than our setup. In turn, our setup supports vision inputs.
There's some speculation that Jev is an open weight model with novel post-training (RLCD). So, if these folks have competitive accuracy with just the base model, it may raise some questions about the necessity of Jev's architecture. You generally don't want to find yourself competing only on price.
I mean just from what's known of the funding and timeline it pretty much has to be based on open weights.
But it is likely more than just a fine tune + novel training. At the very least the LM head is swapped out for a classifier one and then or also idk, bidirectional attention for the encoding pass I'm out of my depth at this point and will stop guessing. The training is probably where they have the biggest moat though, not that it's necessarily huge.
I have a project that fits jev as advertised almost comically well and I've been playing with it, and the various hacks and open versions. Jev doesn't necessarily perform better overall but it is quite different. It's sensitive to prompt phrasing in ways the others aren't, it's easy to generate questions where all the other models cluster in confidence but jev is an outlier. Not necessarily more correct, but it does feel like it's getting its answers in a different way.
I'm guessing just as much as anyone else but I've been spending a ton of time on this the last couple weeks, it landed right when I was most ready to dig into it.
Also, while it's clearly got a lot of training on some use cases, others that probably weren't in the training set have worse good decision rates than a random number generator. If you can rebuild the architecture, you can train it on your use case.
I took a random book excerpt with 23,000 words (±30k input tokens) and used it as context. Jev still responds in 800ms, sometimes 500ms. That's in the neighbourhood of 20-50,000 tok/s prefill, which is obviously not possible with normal LLMs, not even Cerebras is this fast.
i have tested jev for my use cases and its horrendously wrong, but then the follow up from jev's team is "oh, you need to boil the question down further". it's a spiral of how much do you wanna dumb down the ask so that it answers it correctly. i'll pass for now.
We measured latency in separate runs with one request at a time, because timings taken under load measure the queue rather than the model.
As Privatemode is hosted in the EU and Jev is hosted in the US, we ran four of the datasets from Germany and from the US at the same time. From Germany, Privatemode answered in 180 ms and Jev in 264 ms. From the US, the order reverses: 164 ms for Jev against 299 ms for Privatemode.
Is this a joke? “Jev-like” properties? People have been using LLMs as classifiers or rankers in a similar way for ages. I feel like we’re losing our minds
The core idea is to craft the input prompt so that the first output token answers the question. This makes it possible to get a decision with a single forward pass.
In the blog post, we describe the approach in detail for GLM-5.3-Flash and vLLM. We benchmark this setup against Jev and Laya. We find that our setup is on-par with Jev in terms of accuracy and speed and that it substantially outperforms Laya.
Still, in terms of costs per decision, Jev is several x better than our setup. In turn, our setup supports vision inputs.
My question is why not use Jev instead? It's faster and cheaper.
These questions are answered by the OP (Same speed, image support) - additionally, GLM is open weight.
There's some speculation that Jev is an open weight model with novel post-training (RLCD). So, if these folks have competitive accuracy with just the base model, it may raise some questions about the necessity of Jev's architecture. You generally don't want to find yourself competing only on price.
Fyi, I haven't tested this yet.
I mean just from what's known of the funding and timeline it pretty much has to be based on open weights.
But it is likely more than just a fine tune + novel training. At the very least the LM head is swapped out for a classifier one and then or also idk, bidirectional attention for the encoding pass I'm out of my depth at this point and will stop guessing. The training is probably where they have the biggest moat though, not that it's necessarily huge.
I have a project that fits jev as advertised almost comically well and I've been playing with it, and the various hacks and open versions. Jev doesn't necessarily perform better overall but it is quite different. It's sensitive to prompt phrasing in ways the others aren't, it's easy to generate questions where all the other models cluster in confidence but jev is an outlier. Not necessarily more correct, but it does feel like it's getting its answers in a different way.
I'm guessing just as much as anyone else but I've been spending a ton of time on this the last couple weeks, it landed right when I was most ready to dig into it.
Also, while it's clearly got a lot of training on some use cases, others that probably weren't in the training set have worse good decision rates than a random number generator. If you can rebuild the architecture, you can train it on your use case.
Everyone is doing this to emulate Jev, but...
I took a random book excerpt with 23,000 words (±30k input tokens) and used it as context. Jev still responds in 800ms, sometimes 500ms. That's in the neighbourhood of 20-50,000 tok/s prefill, which is obviously not possible with normal LLMs, not even Cerebras is this fast.
was the answer correct?
i have tested jev for my use cases and its horrendously wrong, but then the follow up from jev's team is "oh, you need to boil the question down further". it's a spiral of how much do you wanna dumb down the ask so that it answers it correctly. i'll pass for now.
also, 30k input tokens is a lot.
Isnt this obvious ? I would have thought people would try such things before deciding they need something like Jev
It is.
What isn't obvious is why people keep shouting "Jev Jev Jev" all the time.
Astroturf.
If you are using an autoregressive decoder (which glm is) it is not “jev-like”. You lose all of the speed advantages that Jev has.
turns out there is a trick to keeping the context filled and only evaluating a handful of choice tokens https://www.youtube.com/watch?v=bcGO7xre46o
Right, so it is double the latency and will no longer feel real-time to the end user.
RIP Jev
Is this a joke? “Jev-like” properties? People have been using LLMs as classifiers or rankers in a similar way for ages. I feel like we’re losing our minds