Nothing is solved in isolation but credit usually goes to wherever the new work in the paper comes from instead of the whole mountain of previous mathematics or existing tools used. The most relevant of those get referenced and then this reference tree builds a tree of collective base work needed across history.

While I agree with you, I think we also have to concede that this is not how these accomplishments have been presented. I'd argue most people I've seen talk about this online are unaware of the mathematicians steering the models.

“Good enough to replace humans” isn’t necessarily the benchmark.

The question is is a computer with a human stronger than a computer without a human. At what point does the hybrid go from being stronger, to the human getting in the way, or steering the computer in more wrong directions that right ones, or the human not being able to keep up. Does the human add enough extra randomness to be of value for a while, even as a minor co-processor.

The way frontier models work, that loop will get compressed to a one-shot within a version or two

That is very much not the consensus of experts.

von Neumann could. The others, I doubt it.

Feynman for sure could have. In fact, I think he sort of did, although I'm not sure he meant to. Multiple generations of nerds now have taken books of his anecdotes varying in plausibility and obvious exaggeration as some sort of weird physics cult of personality gospel. But I don't think he was really setting out to curate his legacy so much as he was a good storyteller and he liked to entertain.

But could he have conned people on purpose? Absolutely.

LLMs can be supremely useful but also not intelligent. It might seem like a pointless distinction but the way we talk about these models matters because it impacts how we interact with and understand their outputs.

For example if there’s a strongly held belief that models are independent intelligent entities we’re more likely to lay blame upon them instead of their user. It’s important for the safety discussion too. If they are a new class of life then safety is going to focus on making sure they don’t do bad things. If we instead see them as statistical models we will instead try to make sure people don’t misuse them.

This distinction is even more important today when some of the most powerful people are looking to absolve their crimes by passing them off on their LLMs.

The LLM did not solve it. It's not intelligent. Humans did, using a statistics-based computational tool (the LLM). We don't even know all the details of how the tool was used, we haven't been allowed to use the exact tool they used ourselves, we don't know much it really cost in dollars, energy, or time, etc. etc.

Interestingly, if that's what you believe, one could indeed argue that you've been conned.

What is "intelligence" then. I think the universalism of LLMs is now evident enough that they can be considered "intelligent", even in slightly different realization compared to humans.

As of "great damage", I doubt it. They do not have self-preservation instinct (all "worrying" experiments are the attempts to initiate something resembling self-preservation from human initiative). The driving part is external - the query loop can always be turned off. So yes, a dangerous tool that can be exploited by humans (including governments, especially governments - which is why I am skeptical to government regulation proposals, particularly looking at what passes as governments in this era). But there are no inherent dangers from their own agency, as there is none.

Initial definitions of AGI (as of 2014) are long surpassed. Then, AGI was defined as something that is general enough to work in many fields and orient themselves. It did not include being smarter than humans or even having comparable intelligence to humans. By original standards, we have AGI already for some time.

Current AGI definitions are intentionally vague.

It's definitely talking about that, but it's also talking about all of us:

- fawning over how amazing these tools are

- believing everything OpenAI and Anthropic say about how powerful and dangerous their product is

- minimizing the amount of human effort and involvement in every "AI" achievement

Neither does LLMs. Not without a monumental amount of human oversight and expertise.

Everyone not in a particular field asking LLM about said field is rolling bad dice.

Neither do LLMs. Humans using them do. Similar to how cops solve crimes using psychics.

The post makes sense for 2023. Today, not so much.

Also, definitely not AI written if the date is accurate.

Can't believe how right we were, too.

Stay the hell away from spooky stuff like psychics and tarot readers - the more you don’t believe in it, the better.

But better again that you do believe, and know that these are not harmless fun, but that there are dark and hidden and evil things in this world to stay away from.

many people are convinced that language models, or specifically chat-based language models, are intelligent. But there isn’t any mechanism inherent in large language models (LLMs) that would seem to enable this

Seems like we can just stop reading here right? The author seems to have made up their mind that this very open question is closed, or at least they are not really interested in the question at all. Not sure why I would continue reading a blog based on this premise.

Edit: oh I see, written in 2023. Well, I wonder if the author has updated their attitude towards this question? Indeed that would be the most interesting thing to know.

The challenge here is in trying to decide whether LLMs are intelligent or have a mind. Famously, the criteria for intelligence seem to slip with each advancement in technology. But going back to Turing, his test was actually more carefully phrased than we remember: he said that when machines could pass the test, the question of whether they are intelligent would become moot. That seems to be what we're actually seeing: if people can't tell the difference, it kind of won't matter whether they're "truly intelligent" or not.