https://x.com/dWeaths/status/2102415625301717065 - Here's my use case of jev, being able to accurately detect nouns (with adverbs, adjectives etc.) in realtime as the user is typing it, genuinely feels like it's running locally with how fast it comes back. I've sent over 1000 requests to Jev and its cost me $0.01 (probably rounded up!).
I mean now with modern LLMs a lot of good packages and tools get forgotten about. When you have hammer everything looks like a nail. Even if said "old" tools are actually orders of magnitude faster, and sometimes better too for that specific task. (And I remember spacy being basically SOTA for generalist NLP tasks not that long ago, like 2020/2021).
For my specific example: "the tall dark handsome man wanders into a dark gloomy bar. he orders the biggest beer in the world and sits down on a bar stool surrounded by irish dancers" spacy splits out "the biggest beer" and "the world", whereas I want "the biggest beer in the world" to be the singular noun.
When the user is typing the string, I seperate the last 10 words into reversed joined words, e.g. tall dark handsome man becomes "man", "handsome man", "dark handsome man" and "tall dark handsome man", and I send the full string to Jev and ask it to tell me with probabilities of each option which is the best one to fully capture the descriptive noun
I had an idea for a 'write without space, basicallylikethis' and Jev (given how cheap it is) just being asked after every key press where inserting spaces (or making typo correction) would make sense.
Honestly just getting it integrated into mobile phone swipe keyboards would be a godsend. If I type "We need to get going " and then swipe the word "now", I really do not think "mower" should be the word it chooses. Present a set of swipe-based likely words and the preceding text message to Jev, or similar model, and pick its highest prob word.
i built wellposed as a plugin/skill which works with any agent (https://github.com/suraj-phanindra/wellposed) to ensure your agent understands how to choose the right kind of jev request and format it correctly not just for syntax but for completeness and correctness. it is well-documented in the "jagged-ness" docs (https://docs.typesafe.ai/model-jaggedness/jev-1.13) that typesafe include on their docs page that the absence of essential options can cause jev to choose the wrong option with high confidence (it cannot choose what it cannot see in the request) - so the responsibility to ensure whatever the intent behind your jev request is - it is captured correctly with the right states and options for jev to pick from falls on the user. wellposed should ideally make your agent better at converting NL intent into jev requests. please try and give me feedback. appreciate it!
Don't want to be a conspiracy theorist, but these past two weeks have seen what looks to be a coordinated campaign to boost Jev. Am I out of the loop or is this so revolutionary it warrants getting so much coverage on HN and other professional sites?
It's definitely revolutionary, considering AI news is ~50% of the front page at any given time, and the frontier models are all pretty much doing the same thing and just getting slightly better. This is a whole different technology, it's cheap and fast, and it's usefulness at different tasks is still being established. Amazing for hackers!
JEV directly addresses many of the most common issues with LLMs for certain applications. It seems IMO to be overhyped right now but I think it may, like the broader llm ecosystem, be here to stay.
There's definitely a level of inauthenticity in the hype, but that's a function of the times we're in.
Revolutionary? With an appropriate harness could could do the same thing with the vast majority of modern large language models.
It's just so much faster and so much cheaper that it feels qualitatively different.
I also think it represents a bit of validation for folks looking for ways to bake models into hardware. Sometimes it's perfectly appropriate to sacrifice good for fast and cheap. I haven't asked Jev to do anything that GPT-3.5 would have likely done worse with.
Initial personal use case (on my side project https://mealplannr.io/lists) is lists have a "smart categorise" button to group items into 20 or so preset lists.
Previously this took up to ~30-60 seconds using deepseek v4 flash (even with a medium list size) - Jev is <1 second @ same cost with typesafe guarantee
As a bonus I can also instantly categorise new items - rather than sending them into an "unknown" category (and waiting for user to have to click "categorise" again)
At this point, I am finding it extremely hard to believe that Jev team is not on a massive astroturfing campaign. This is happening all over reddit too. All LLM subreddits are getting flooded by Jev posts, many of which are made by new accounts that only talk about Jev, many obviously advertising in guise of sharing knowledge (e.g. https://www.reddit.com/r/LocalLLaMA/comments/1wn4cni/removed...)
Multiple posts on HN, including this one, are from accounts that only ever talked about Jev. Each get unusually high number of upvotes early on, enough to put them on frontpage. A multitude of commenters on such posts also seem to talk about only 1 topic.
Can all this happen organically? Yes but with vanishingly low probability, from my vantage point.
"A compelling clip often leaves the useful questions unanswered: what did Jev decide, where is the implementation, and what can I reuse?"
Oh my god for the love of god and all that's precious please stop using Claude. Just reading this makes me want to set up a swarm of rogue agents to break into anthropic and fix this writing. I hope this changes!
Basically compared to standard LLM models it is an order of magnitude cheaper and faster. (Note: I didn't get to actually try jev yet, just looked at demos/specs/pricing etc)
You can definitely do similar things with say hosted LLMs + a lib like outlines , or with API models and the proper output validation layer, but again way slower and more expensive.
And on the opposite side you can train dedicated classification models that will be even cheaper and faster to run than jev. But, well, you need to train them (costly, time consuming, and data might be hard to come by depending on target). Here you are a nice zero-short system, that can handle complex/messy data out of the box.
Thanks. How reliable is the world knowledge? Say, I use it as a delegation router for picking the best model for a task - how can I be confident it's doing that with enough intelligence?
When I use traditional LLMs I get some sense from the flagship-ness and regular usage. How do we get such confidence for Jev like models?
The Jev alternative, laya is only 334m parameters and you can post train it as the workflow is open source. The downside is that laya is only 1024 tokens, and Jev is ~32,000 tokens. I think for many tasks 32k tokens might be too small. You could probably design a task complexity router with something like qwen 0.8b with similar speed and a much larger context window (260k token). As always, "it depends". 0.8B is almost as fast as Jev without any of the limitations.
https://x.com/dWeaths/status/2102415625301717065 - Here's my use case of jev, being able to accurately detect nouns (with adverbs, adjectives etc.) in realtime as the user is typing it, genuinely feels like it's running locally with how fast it comes back. I've sent over 1000 requests to Jev and its cost me $0.01 (probably rounded up!).
I'm curious why you didn't use a spaCy for this. It would probably be faster, and you'd avoid a third party API dependency.
I mean now with modern LLMs a lot of good packages and tools get forgotten about. When you have hammer everything looks like a nail. Even if said "old" tools are actually orders of magnitude faster, and sometimes better too for that specific task. (And I remember spacy being basically SOTA for generalist NLP tasks not that long ago, like 2020/2021).
For my specific example: "the tall dark handsome man wanders into a dark gloomy bar. he orders the biggest beer in the world and sits down on a bar stool surrounded by irish dancers" spacy splits out "the biggest beer" and "the world", whereas I want "the biggest beer in the world" to be the singular noun.
So, you send Jev a string of text and it sends back what? I thought it only output probabilities of a classifier.
When the user is typing the string, I seperate the last 10 words into reversed joined words, e.g. tall dark handsome man becomes "man", "handsome man", "dark handsome man" and "tall dark handsome man", and I send the full string to Jev and ask it to tell me with probabilities of each option which is the best one to fully capture the descriptive noun
Did anyone do prompt injection detection?
I wrote a generic harness you can try it out with publicly. Does well with at least simple prompt injection:
https://gpu.studio/jev
I had an idea for a 'write without space, basicallylikethis' and Jev (given how cheap it is) just being asked after every key press where inserting spaces (or making typo correction) would make sense.
The demo is here: https://levmiseri.com/nospace
This one is cool as hell
And so, I typed "Thisoneiscoolashell", and got "This one is cool a shell" hahaha
Great idea, and it's really quick
Thanks! Made some updates to it to strengthen the corrections/splits. Should work substantially better now.
Honestly just getting it integrated into mobile phone swipe keyboards would be a godsend. If I type "We need to get going " and then swipe the word "now", I really do not think "mower" should be the word it chooses. Present a set of swipe-based likely words and the preceding text message to Jev, or similar model, and pick its highest prob word.
[flagged]
"has_happened": { "type": "noul", "noul": 0.96 }
I built hn4me.xyz - Hacker News stories curated by Jev based on your interests
Demo here: https://x.com/RaahelSaidWhat/status/2102162969656475973
Chrome extension to filter Twitter posts for AI slop, promotions, or based on custom rules - https://chromewebstore.google.com/detail/clean-twitter/oflan...
https://github.com/midplane/clean-twitter
i built wellposed as a plugin/skill which works with any agent (https://github.com/suraj-phanindra/wellposed) to ensure your agent understands how to choose the right kind of jev request and format it correctly not just for syntax but for completeness and correctness. it is well-documented in the "jagged-ness" docs (https://docs.typesafe.ai/model-jaggedness/jev-1.13) that typesafe include on their docs page that the absence of essential options can cause jev to choose the wrong option with high confidence (it cannot choose what it cannot see in the request) - so the responsibility to ensure whatever the intent behind your jev request is - it is captured correctly with the right states and options for jev to pick from falls on the user. wellposed should ideally make your agent better at converting NL intent into jev requests. please try and give me feedback. appreciate it!
marketing blitz in progress
Don't want to be a conspiracy theorist, but these past two weeks have seen what looks to be a coordinated campaign to boost Jev. Am I out of the loop or is this so revolutionary it warrants getting so much coverage on HN and other professional sites?
It's definitely revolutionary, considering AI news is ~50% of the front page at any given time, and the frontier models are all pretty much doing the same thing and just getting slightly better. This is a whole different technology, it's cheap and fast, and it's usefulness at different tasks is still being established. Amazing for hackers!
JEV directly addresses many of the most common issues with LLMs for certain applications. It seems IMO to be overhyped right now but I think it may, like the broader llm ecosystem, be here to stay.
There's definitely a level of inauthenticity in the hype, but that's a function of the times we're in.
Revolutionary? With an appropriate harness could could do the same thing with the vast majority of modern large language models.
It's just so much faster and so much cheaper that it feels qualitatively different.
I also think it represents a bit of validation for folks looking for ways to bake models into hardware. Sometimes it's perfectly appropriate to sacrifice good for fast and cheap. I haven't asked Jev to do anything that GPT-3.5 would have likely done worse with.
The fly brain thing on the other hand.
Initial personal use case (on my side project https://mealplannr.io/lists) is lists have a "smart categorise" button to group items into 20 or so preset lists.
Previously this took up to ~30-60 seconds using deepseek v4 flash (even with a medium list size) - Jev is <1 second @ same cost with typesafe guarantee
As a bonus I can also instantly categorise new items - rather than sending them into an "unknown" category (and waiting for user to have to click "categorise" again)
At this point, I am finding it extremely hard to believe that Jev team is not on a massive astroturfing campaign. This is happening all over reddit too. All LLM subreddits are getting flooded by Jev posts, many of which are made by new accounts that only talk about Jev, many obviously advertising in guise of sharing knowledge (e.g. https://www.reddit.com/r/LocalLLaMA/comments/1wn4cni/removed...)
Multiple posts on HN, including this one, are from accounts that only ever talked about Jev. Each get unusually high number of upvotes early on, enough to put them on frontpage. A multitude of commenters on such posts also seem to talk about only 1 topic.
Can all this happen organically? Yes but with vanishingly low probability, from my vantage point.
If it's an astroturfing campaign it's very likely jev powered.
Which, comes to think of it, if true, is self-reinforcing once discovered.
Same thing on Twitter, I noticed last night. Making extraordinary, categorical claims too.
"A compelling clip often leaves the useful questions unanswered: what did Jev decide, where is the implementation, and what can I reuse?"
Oh my god for the love of god and all that's precious please stop using Claude. Just reading this makes me want to set up a swarm of rogue agents to break into anthropic and fix this writing. I hope this changes!
Can someone ELI5 me why Jev class models matter?
Note: Not the technical side, but as an end user of LLM APIs.
I was wondering about the same and found this video very helpful. It may not be truly ELI5, but close enough: https://www.youtube.com/watch?v=tTnUcSj-QPA
That helps, thanks.
Basically compared to standard LLM models it is an order of magnitude cheaper and faster. (Note: I didn't get to actually try jev yet, just looked at demos/specs/pricing etc)
You can definitely do similar things with say hosted LLMs + a lib like outlines , or with API models and the proper output validation layer, but again way slower and more expensive.
And on the opposite side you can train dedicated classification models that will be even cheaper and faster to run than jev. But, well, you need to train them (costly, time consuming, and data might be hard to come by depending on target). Here you are a nice zero-short system, that can handle complex/messy data out of the box.
Thanks. How reliable is the world knowledge? Say, I use it as a delegation router for picking the best model for a task - how can I be confident it's doing that with enough intelligence?
When I use traditional LLMs I get some sense from the flagship-ness and regular usage. How do we get such confidence for Jev like models?
The Jev alternative, laya is only 334m parameters and you can post train it as the workflow is open source. The downside is that laya is only 1024 tokens, and Jev is ~32,000 tokens. I think for many tasks 32k tokens might be too small. You could probably design a task complexity router with something like qwen 0.8b with similar speed and a much larger context window (260k token). As always, "it depends". 0.8B is almost as fast as Jev without any of the limitations.