Grammars do risk pushing models off distribution in a way that impacts their output quality in a way Jev allegedly does not suffer from. Additionally, Jev's ability to answer questions independently is also exciting. Using an LLM to answer multiple questions in one generation has the property of earlier answers influencing later ones. TBD how many of TypeSafe's claims stand up, but my testing so far is promising. I hope they author some papers on their methods as well, but that might destroy their moat.

Not an expert at all here, but I saw a comment on the jev post saying that it you constrain an LLM suck that it outputs a valid structure, if the token with the highest probability is not the one that you expected because of the structure (and so you pick the valid lower one) this means the LLM was already confused and your answer is less likely to be correct anyways.

Is this actually true ?

It seems that anyway most of the value is in the speed and cost.

If it really matters to you whether whether some business-specific classification confidence is above/below some specific threshold (vs just relative order), then you'd be better off training or fine tuning a custom model for that. Maybe that is something that TypeSafe are planning to also provide?

I think the idea is that the latent thinking space in the LLM will be roughly the same for similar quality results - so the majority of executing well could be stripping back and fine tuning an existing LLM.

The claim of how they are doing it is likely wrong.... if you had to bet, it's likely an encoder model of some sort.

The more likely scenario is either OpenAI or Anthropic just go pay some highly inflated price to buy Jev. Mainly just for its people and the PR, not tech.

Which is likely what all the VC, hype machine, and overinflated claims are really about anyways.

The tech etc is easily replicated. The hype / name, not.

I seem to remember reading that the Jev-founder-guy is ex-OpenAI anyways. So that's how these things often roll.

{ "answer": "might", "probabilities": { "might": 0.99, "won't": 0.01 }, "confidence": 0.99 }

ASI isn't here yet. It could benefit people in the interim and help make the ASI.

the point of this article perplexes me. the implication is we should all just take our Quietus. you decide.

I would assume that they would fold this type of classification into their responses API next to existing ancillary tooling that they already ship. I think OpenAI positions themselves as being wholly focused on AGI - but I think their business model leans towards lock-in via superior tooling (Codex, ChatGPT, etc.). This feels like an easy win for them without muddying the larger vision.

without getting too far into it i would just like to note that i am in both these circles

There's a big difference between what AGI can accomplish and what it will accomplish. We must also ask why OpenAI wants to summon their God, and how they would want the average person's life to change if it comes. I think they will try to clone Jev just to add another revenue stream.

Well sure, in the future you may also be able to ask AGI to "please just run my life", while you stay in bed.

In the meantime, today, in the real world, there are businesses wanting to automate well-defined business flows, who don't want some stroppy AGI with a mind of it's own to instead decide to hack into something, or reward hack and make the customer happy by just wire transferring $1M of company money into their account.

Why would it need to? There is a deterministic flow here for the actions that are allowed. AGI isn't needed for this at all if you can map out the flow and use a classifier to decide which route to follow.

I don’t think that was a great example since there’s only so many refunds a customer is going to ask for. And it’s saving time that otherwise maybe would have to go through a human. The rate is low enough that a more expensive model makes more sense.

Though for tasks where you are trying to search through billions of documents, social media posts, etc. and extract certain information, where each individual post is of low value and only the data in aggregate is valuable, then that’s where you’d want something cheaper and faster.

Such as if you want to look at all posts on X in the last few months and find how many have a negative or positive sentiment about the economy (or are unrelated).

Of course you could use a special-purpose model for this, but the whole point of something like Jev is to ask whatever questions you want without having to train something new.

I didn’t realise they were competing. If anything Jev seems complimentary to LLMs.

A lot of business automation doesn't need AGI, doesn't want to pay for AGI if they don't have to, and would be better off using a classifier than something increasingly unreliable with a mind of it's own.

There are many automation pipelines that use LLMs because there was no choice, but the multi-way classification that Jev provides is exactly what they need, and is going to be way faster and cheaper, as well as having the benefit of calibrated probabilities and structured output that can be relied on.

Exactly, the future is open models. Which is also the reason why those overvalued companies such as OpenAI will lead to a market crash as soon as investors realize that.

That's not what I meant. I'm thinking more of AI systems that combine multiple modules the same way the brain has different regions. LLMs are just part of the bigger picture here. They're good at tackling a certain types of problems, but other approaches are better for different kinds of problems. Having a system that combines a generative model and a classifier for example would make it a lot more efficient and accurate because it has a bigger toolbox instead of using one algorithm for every problem whether it fits or not.

It's not just OpenAI. It can be any frontier-level lab that has more funding than Jev.

this one feels closer to the claw cycle

Yeah but they weren't that great, you couldn't ask for arbitrary classifications after the model was trained. You are underestimating what they've done here, even if it does seem a little overhyped.

In my company, and I think in most companies that are using AI at all, one of the first ways it got integrated is as a classifier, to tag orders based on feeding all their data into a prompt and asking for a structured output.

I think demand for tools that are more tailored for this type of integration is high. I don't really understand why Jev is supposed to get my company's decisions right more than an LLM, but regardless of the tech I think people are just excited about the possibility of iterating faster, more explainability, higher-level tools that are specifically created to help hone classifiers etc.

Beats me also - this feels unreliable, extremely niche, and over-hyped. I don't trust LLMs even when they explain their reasoning; the idea of trusting a black-box classifier like this seems insane.

With all the excitement around Jev, I suspect we'll see hundreds of options, it doesn't sound like Jev is that hard to replicate, given all the 3rd parties who are getting pretty damn close, or even better, results within a week.

vLLM has a PR very close to merging: https://github.com/vllm-project/vllm/pull/57250

Kev is an open Jev: https://github.com/jaredpalmer/kev

maybe open source even eats Jevs lunch first

see here: https://news.ycombinator.com/item?id=49800574

Or OpenAI just buys them outright. Buying your upstart competitor seems to be in the Silicon Valley Ten Commandments. The Fed whussed-out on breaking up FB and Insta last year, so there's never going to be any kind of remediation to worry about.

And for Jev, everyone has a price, and OpenAI's raised an historical amount of funding.

Which clause? For instance openrouter lists openai as "Logs: this provider may retain prompts, but does not use them for training."

https://openrouter.ai/openai/gpt-6-astra#providers

I have been told that ZDR is available upon manual request.

At this point, we shouldn't trust company even with ZDR.

Keeping people looped into your product is pretty important, but yeah, there's not clean way currently to separate "structured" outputs from the token stream and to start using a different billing structure there. And I also appreciate that they aren't going to be keen on gving free or near free output either, so gotta figure that.

system 2 is just an llm with a forced toolcall IMO

In the olden days we call this classifier, usually assignment 2 of Machine Learning 101. BERT (well, GLiNER specifically) and diffusion are calling and want their Large Classifier Models back.

https://github.com/vllm-project/vllm/pull/57250

I just had Claude and Jev combine forces last night. I've built a few personal browser extensions in the past and thought it would be fun to copy an experiment I saw on twitter where Jev classifies comments/posts etc as slop or not.

Fed Claude an api key from typesafe and a link to documentation, and within about 10 minutes I had a view of HN that was populated with a little ranking as to sloppiness of each comment.

When your mind has been wired a bit to LLM latency, it feels extremely fast, and for such a subjective rating I think it did a good job.

Feels like it sits in a space between traditional ML classification and the frontier models. I can't think of a 'real' production use case for it in my sphere of influence, but certainly some will. And of course there will be five Jev competitors by the end of the year.

I'm glad someone noticed :P - It was hilarious once we figured out what was going on. We had literally removed it's ability to shut up.

Not directly related, but still jev related:

Would it be intesting/useful to use jev to generate a block of text like LLMs do ?

Like asking it to pick the n + 1 word given the starting text (using it's choice primitive), but also asking n + 2,n+3 and so on at the same time.

Would it give coherent or useful results ? Or does the fact that it computes it "all at once" means it cannot make one of it's answer influence the other ones ?

Jack of all trades, master of none.

autoregressive LLMs are indeed generating “probabilities”

I find it slightly more helpful to say they generate plausibility

I don't think it's unreasonable to think this, but I do think the burden of proof is on your side. Between the SaaS-pocalypse narrative that never materialized, and inexplicably losing their first-mover advantage to Anthropic, OpenAI's track record is not great when it comes to jumping on these micro paradigm shifts.

If the headline said "Frontier labs are about to eat Jev's lunch" it might be an easier sell. But if we're gonna include Anthropic, I think part of their success is actually making products for which there is demand. It will take time for something like that to come out of this new "decision model" paradigm.

This article is extraordinarily hard to read. It’s tummelvisioned on OpenAI and things like tool calling which are only relevant to the extent that llms have been tuned to make relative choices, but this applies to all LLMs. Also, some really outdated references. LLM written, perhaps?

Also, moat discussion is the lowest form of discussion. I don’t care if jev has a moat. Did it get the interface right? What other past ideas have we overlooked that if given some love, could kick the door down like jev did?

Really silly stuff.. people wanting to talk about moats when there’s no castle. Moat talk merely projects the illusion of being engaged but, much more often than not, it’s hollow engagement.

chaotic neutral

Nope. Here's the closest I got https://arcturus-labs.com/blog/2025/03/31/supercharging-llm-... - 1.5 years ago! But I never really did anything with it. And I wasn't thinking about reinforcement learning anything.

Jev certainly feels vulnerable but whether it’s oai or someone else that goes after them seems unclear.

Wouldn’t be surprised if every single AI house spins up a copy

But like they usually also have an embeddings endpoint

Asking again (didn't get an answer in prior discussion):

As there have been a lot of Jev related submissions, can someone point me to a simple guide on how I can use it? For example, say I have a script/workflow where I use OpenRouter for LLM calls, and at some point I want to do a simple classification. Can I still use OpenRouter with some Jev model...?

Even the article itself has the title as a question: "Will OpenAI eat Jev's lunch?". A more useful title would be "OpenAI is Positioned to Compete with Jev".

For context on what "Jev" is: https://news.ycombinator.com/item?id=49717558

You can easily use Jev without an LLM (consider Jev used to make truthy decision branches in a script). It's not meant to replace LLMs. The tech is not a competition. But OpenAI is certainly in competition with TypeSafe, they'll want to keep people in their own ecosystem!

Rather than focusing on OpenAI in particular, let's just say that there are many smart people at other AI labs and if it seems like it will be popular, this technique will probably be copied. What would prevent them from adding another API?

Hopefully there will be some decent benchmarks and gateways for switching between providers easily.

For me, I think the big deal is that it promises to be general and broadly applicable and high quality. That's new and special. But we'll wait to see if the claims actually hold.

I fed into the hype at first. Testing Jev and Laya, they both suffer from the same issues as LLMs that stop them being useful beyond limited classifications.

I can't see any benefits that a typical ML classifier would not be better at.

Yeah, if your "master AI" is good enough, it should be able to find and learn about and use specialized tech AI like Jev if it suits your goals, before you even know about such stuff.

You might find my old post interesting https://arcturus-labs.com/blog/2025/03/31/supercharging-llm-...

That article is a bit myopic. People and companies don't want to feed all of their ideas, projects, and intellectual property to a morally unscrupulous oligopoly. We learned that lesson the hard way with the last batch of tech monopolies, and the shift towards majority open weight models proves the trend.

Open weight classifiers and open weight LLMs will be burned onto silicon cards in a few years after the models begin to stabilize. They'll be in PCs and laptops. That's going to capture a HUGE chunk of the market.

If you need more horsepower, you'll rent the same silicon safely from AI services cloud providers without handing your data over to Anthropic and OpenAI.

their pipeline would be to just prompt it's internal next gen models to create a jev copy given all the data they have as a first pass.

I’m shaking my head in disbelief.

Reading logits is the cornerstone of ML. It’s almost like many of the people reporting on and fawning over AI have no technical background and never knew about ML classifiers or calibration.

The OpenClaw vibes are hard to miss here.