- 111comments
- 197comments
- 15comments
- 42comments
- 8comments
- 36comments
- 63comments
- 217comments
- 12comments
- 234comments
- 4comments
- 143comments
- 40comments
- 83comments
- 20comments
- 41comments
- 167comments
- 14comments
- —discuss
- 116comments
- 10comments
- 85comments
- 138comments
- —discuss
- 8comments
- 23comments
- 236comments
- 15comments
- 16comments
- 74comments
Apparently once you add a "Grading criteria" text, the model collapses into a "compressed toward the center of the scale" hallucination (or training set overfitting).
Someone on X asked me to try to reproduce it, and I actually got it on the first try on their Gemini Chat:
https://x.com/XCSme/status/2057613611959279988
I am not sure what to make of this (or most SOTA) models. They got a lot smarter with coding and tool usage, but a lot dumber in other ways...
Direct link to the chat, ignore the story, it's just some filler tokens: https://gemini.google.com/share/244af1e74841
To save you a click, this is the output:
What happens if you prompt, "don't compress the scale"?
lol, same result but without the explanation on why:
https://gemini.google.com/share/96f93fe601c6