- 62comments
- 165comments
- 656comments
- 73comments
- 20comments
- 11comments
- 141comments
- 41comments
- 24comments
- 4comments
- 2comments
- 13comments
- 101comments
- 341comments
- 61comments
- —discuss
- 65comments
- 569comments
- 2comments
- 30comments
- 1comments
- 200comments
- 77comments
- 14comments
- 25comments
- 1comments
- 7comments
- 7comments
- 137comments
- 287comments
I wonder when work started on this project, and how the public release of Jev played into their timing.
From a tweet:
https://x.com/jackyk02/status/2102947585208258799?s=20
But of course the repo with the TypeSafe compatible API is new: https://github.com/Contrastive-LM/CLM
Does the latency even matter?
You're comparing a local GPU to network hops? Wouldn't be surprised if Jev was actually similar in runtime and their is just a great deal of network latency.
The evaluation is quite interesting though - I'd actually say the raw answer is correct in the absence of detail and prior knowledge (Who wrote the play Romeo and Juliet).
I tried TypeSafe’s Jev playground. It outputs the model latency and network latency separately. The model latency was 100-200ms in my tests.
what's with that dino run? Jev is slow but it jumps correctly, their model always touches the cactus or whatever it is...I am guessing it doesn't matter? Or does it?
Very interesting insight on the training process, it's pretty cool to have some experimental justification for why they took these exact steps, what they tried and did not work, etc. Feels a bit less like dark magic.
However I agree the latency argument doesn't hold much value with Jev because it runs on a remote server. Seeing how many open Jev-like models came out recently it would be much more interesting to have a comparison with them.
I really hope that "System One" won't stick around as a new buzzword simply meaning "fast".
It already did, since 15 years ago.
https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow
i'm well aware of the origin and meaning of the term
this implies more than just "fast", which is precisely why i don't like its present usage
Instinctive is a good way to describe it compared to generative LLMs. Jev gives you one instant answer, fast and usually correct but without nuance or any explanation. Human instincts work the same way.
I tried running this on a H100 and got 190ms compared to Jev's 170ms. Maybe I set it up wrong?