I tested Jev with a fair die 400 times without telling it the die result. The true probability of face 1 is 1/6, but Jev always chose face 1 and the probability it returned was about 83%. I also tested with a fair coin 200 times and got 0.92 probability.
I did several tests and I think Jev is good at problems with a correct answer but weak at problems about actual probabilities whose answers can't be known at all.
But "problems about actual probabilities whose answers can't be known at all" are exactly the problems where calibration is important. Since calibration is one of the big claims about Jev I'd expect it to perform well in these problems.
I agree. I think it's odd behavior too. Jev should be good at actual probability problems given the phrase "calibrated probabilities" TypeSafe uses for Jev. Maybe the reason is the data used in their training method (RLCD). If all the data consists of problems with a correct answer, I think this kind of odd behavior could happen.
The reason and the word you're looking for is mode collapse. It happens during RL, it's not a dataset problem, and it's the reason for many issues with LLM writing and all sorts of issues in all types of models. The result you got is 100% predictable and unsurprising, as it's a neglected problem nobody really rushes to solve as current big model training is laser focused on one thing and one tool.
Hah! I did the exact same tests as you! I found that if you give it the choice to say "not sure", it picks that 100% of the time. But if you pin it in a corner, then yes it does these weird things. Also yes, the continuous options were much more accurate than the choices. Not sure why that is.
Fine tuning LLMs has turned out to be mostly not worth the effort, but I wonder if fine tuning Jev-style models will turn out to be a whole lot more useful.
I’m using Jev to classify a blob of text I see in browser with an extension. Calibrated? No. But it’s handy enough. If a large blob of text is likely AI generated, I’m very likely to skip it
I tested Jev with a fair die 400 times without telling it the die result. The true probability of face 1 is 1/6, but Jev always chose face 1 and the probability it returned was about 83%. I also tested with a fair coin 200 times and got 0.92 probability.
I did several tests and I think Jev is good at problems with a correct answer but weak at problems about actual probabilities whose answers can't be known at all.
Write-up: "Jev Does Not Play Dice" https://kantahayashiai.github.io/posts/jev-does-not-play-dic...
But "problems about actual probabilities whose answers can't be known at all" are exactly the problems where calibration is important. Since calibration is one of the big claims about Jev I'd expect it to perform well in these problems.
I agree. I think it's odd behavior too. Jev should be good at actual probability problems given the phrase "calibrated probabilities" TypeSafe uses for Jev. Maybe the reason is the data used in their training method (RLCD). If all the data consists of problems with a correct answer, I think this kind of odd behavior could happen.
The reason and the word you're looking for is mode collapse. It happens during RL, it's not a dataset problem, and it's the reason for many issues with LLM writing and all sorts of issues in all types of models. The result you got is 100% predictable and unsurprising, as it's a neglected problem nobody really rushes to solve as current big model training is laser focused on one thing and one tool.
Echoes a bit of a philosophical distinction with a long history: "Knightian Uncertainty" versus "Probability".
Hah! I did the exact same tests as you! I found that if you give it the choice to say "not sure", it picks that 100% of the time. But if you pin it in a corner, then yes it does these weird things. Also yes, the continuous options were much more accurate than the choices. Not sure why that is.
If you instead offer probabilities as answers, it picks the right one with high credence.
Maybe I’m confused here, but it’s perfectly reasonable to just guess the same dice roll every time right?
In future, we will see intiatives similar to OpenStreetMap for Textual data or Web similar to high quality non-contaminated steel.
I'm really looking for a multi-modal image capable version of Jev.
If we could get machine learning type results on images without training, that would be fantastic.
Fine tuning LLMs has turned out to be mostly not worth the effort, but I wonder if fine tuning Jev-style models will turn out to be a whole lot more useful.
I’m using Jev to classify a blob of text I see in browser with an extension. Calibrated? No. But it’s handy enough. If a large blob of text is likely AI generated, I’m very likely to skip it