It's the digital equivalent of Morty speaking with the death crystal: https://youtu.be/YjepJlvkdKs?t=51. The crystal shows him how he will die, so he iteratively determines his speech based on whether he sees himself dying with the life he wants.

> write me a short story

a story

I think this is the first time I’ve knowingly laughed at a model’s joke.

Reminds me of the very early LlMs. I also sometimes miss the sheer demented horror of early image models told to generate images of biology.

Codex and I were trying to infer why Jev gave a certain parameter a given score. So I asked codex to produce a list of like 30 plausible reasons Jev might've selected that and then presented Jev with the initial prompt, followed by "You scored this with XYZ. What was your reasoning for doing this?" And then allowed it to do a noul value for each of the reasons codex generated. Felt like those people that give their dogs the buttons to push.

Jev has taught me the same lesson three times over now.

When it first came out, I thought "this weekend, I'll do a little open-source Jev based on single-token prediction and the token logit output", but of course when it came to it, there were at least 5 that had already been done between me thinking that and getting around to it.

So I wrote up[0] what other people had done, but wasn't happy with how weak the benchmarks were, but in the time between writing the first word and the last few, two excellent sets of benchmarks had been written, so I was able to incorporate those. I published the article, and one of the authors of one of the implementations commented that I'd beaten him to doing the write-up he'd wanted to.

This morning I thought "huh, you could have some fun giving Jev a single letter or token at a time, turning it into a chatbot", but as the time of looking two people had already done this (and taken the gag further than I would have), and ... this is isn't either of the ones I'd found. I bet if you scratch the surface there already at leat 5.

Time from idea to output has dropped off a fucking cliff.

0: https://sgnt.ai/p/jev/

Time from pointless idea to bad output, anyways. We aren't seeing good software, and now neat hobby project ideas are getting harvested pointlessly when the only purpose of those ideas was the fun and learning of doing.

LLMs are now like major highways, and everyone thinks theyll solve software jams by just adding one more lane; but that just induces demand, and doesnt increase efficiency because the traffic jam is about how people evaluate usage and fill the voids.

Similar to how we upgraded computers for decades and the software bloated to fill the specs

Good ideas are the survivors of lots of bad ideas.

Slack is glorified IRC yet they're worth billions.

Dropbox can be trivially implemented via rsync yet they're worth billions.

There is a canonical hacker news post about Dropbox being useless and a pointless idea at one point :)

I had the same sort of thing going on ahah. But I was convinced some hoje must have already done it and I decided I’d research when I got home (I’m out today). Didn’t expect it to reach hacker news so soon, though!!

I'll do a little open-source Jev based on single-token prediction and the token logit output

How is this idea generally working out in comparison with Jev? I'm curious, from what I read so far it seems like Jev is still beating this kind of thing.

But it's curious because it's not entirely clear why, from an architecture point of view for all we know that's exactly what they're doing. So it must come down to the quality of those logits, ie., model size and training details.

It seems to me that what most of these single-token-prediction projects are missing is that Jev seems to be claiming they predict well-calibrated probabilities. This is an incredibly valuable thing that LLMs simply can't deliver unless they are trained specially for it.

i think the next step is make Jev a emoji bot... The architecture and its limitations would work well in that regime imo better then human language.

I think it would work well as a multi-emoji compiler too. Some phrases pair well with N emojis, and you’d be able to produce arbitrarily many emojis per phrase

I saw a lot of ppl think about what jev could use under the hood and could someone explain why this can't just be an embedding model where we just embed all the input + decisions and give back the cosine (or whatever) similarities?

if you compare an embedding model to something like Jev which asks 100 questions and use the answers as the embedding you will be able to get move mileage out of the latter. especially because you don't need to train any classifiers for your task, you can work directly on the answers.

that said, I don't understand the hype. I have been doing what Jev does for 2 years now by just forcing json tokens onto an LLM. you can even get the LLM to think. and you can ensemble multiple LLMs.

I suppose the appeal of Jev is how cheap and fast it is, but then it's entirely unsuitable for anything but the most cursory extraction. using it to play games seems like a waste of time especially when most of those games will be played better by an algorithm written by an LLM (just give it the state and ask it to write a bot).