I actually explored this for robotics last year, and CLM grew directly out of that work a few months ago. Feel free to check out my earlier post for more details on how CLM could be applied to robotics:
https://x.com/jackyk02/status/2026368947210289660
You're comparing a local GPU to network hops? Wouldn't be surprised if Jev was actually similar in runtime and their is just a great deal of network latency.
The evaluation is quite interesting though - I'd actually say the raw answer is correct in the absence of detail and prior knowledge (Who wrote the play Romeo and Juliet).
I wonder when work started on this project, and how the public release of Jev played into their timing.
From a tweet:
https://x.com/jackyk02/status/2102947585208258799?s=20
But of course the repo with the TypeSafe compatible API is new: https://github.com/Contrastive-LM/CLM
Does the latency even matter?
You're comparing a local GPU to network hops? Wouldn't be surprised if Jev was actually similar in runtime and their is just a great deal of network latency.
The evaluation is quite interesting though - I'd actually say the raw answer is correct in the absence of detail and prior knowledge (Who wrote the play Romeo and Juliet).