Yea, it's kind of odd how "sour grapes" people can be when something stops being as special as they thought it was.
Take someone from a few hundred years ago and drop them into today, and if they don't go catatonic and die, then they'd tell you that we created magic. "Wow, you live in a world of magic and all you do is bitch about it".
"You're flying! You're sitting in a chair, in the sky!"
Actuallllly these hypothetical people from a few hundred years ago hypothetically said you are an awful person for misrepresenting them, stop putting words in their mouth, thank you!
A person inventing something out of dumb luck is interesting.
A machine running a loop through an expensive LLM for an undisclosed amount of time, which cost an undisclosed amount of money, which was told to keep looping into a solution was found, for a problem that nobody was very concerned about... That just seems like PR, and it's not so interesting.
I believe an LLM can solve pretty much any problem for which we can define a fast iterative loop and for which we have reliable tools to automatically verify the correctness of a result. That’s how they are able to solve some math problems, and how they are able to generate working code. Then it’s a question of how much you have to pay for the model to explore the space of solutions in a reasonable timeframe
In math a lot of the spectacular results have been made by finding a counterexample or worming their way toward a proof that is very well defined.
There's some debate over to what degree current generation AI can be creative at all, or whether it can only crawl around its latent space and explore within constraints. One might ask: were all the solutions to all the math problems AIs have solved already "there" latent in the training data and just hadn't been spotted by humans and put together?
But then... isn't everything latent in our training data if training data is "all observations made about the universe?"
But then... what even is creativity? That gets into philosophy and metaphysics. Creativity, like consciousness and sentience and self-awareness, is not a rigorously well defined concept. So to a degree we don't even know how to ask the question of whether these things are creative.
It's easier to evaluate certainly. Did it solve the problem? Yes/No
When you design a new thing it will have a dozen drawbacks and a dozen and one benefits. If people then have a bias that everything ai is bad, the signal won't be strong enough to convince.
This dichotomy is counter-productive. The valuations floating around are insane, and the claim that some software can replace every single laborer is one step removed from fiction. At the same time, this stuff is clearly going to change how to world works in countless, deep ways. But the idea that some fancy autocomplete can replace humans ignores the reality of humanity and the fancy autocomplete.
It’s another dot com bubble, not a crypto bubble. Trillion dollar valuations burst once you leave lesswrong.
Edit: very fancy autocomplete. I know what these things are capable of. It’s still not “intelligence”, for X definition of intelligence. It is awesomely impressive synthesis of data, yet it’s clearly still that.
i dont think humans are going to be replaced either, but calling the thing that solves millennium problems and is currently changing multiple industries entirely a "fancy autocomplete" makes it harder to take any point you are making seriously.
The SS-Totenkopf Division was advancing east during the opening weeks of Operation Barbarossa, the German invasion of the Soviet Union. 10 July 1941, the division had just fought its way through the Soviet border defenses around Sebezh. It had moved through Lithuania and Latvia, crossed the Dvina area, and advanced through Dagda toward a place German records called "Rosenow." The division moved out of the Rosenow area around 6 July, fought around Sebezh on 8-9 July, and then continued east/northeast toward Opochka and eventually Porkhov.
Do we have any kind of transcript as to how the message was cracked, and whether this was cheaper or more expensive than simply Bombe-style trying all the combinations?
Phew, never have I seen in my life the goalposts move so fast.
It seems like even yesterday that the threshold for impressing someone is that the machine would have to be good at pretending to be a person. Now the threshold is that they have to be able to invent special relativity.
Even the Millennium Prize Problems have, in a way, become benchmarks for model companies to prove themselves. The smartest individuals among humans are becoming replaceable. Intelligence has become a product you can quantify and buy with electricity. That feels awful.
Even those of us that are pro AI need to acknowledge this is the current reality.
The smartest humans now need to move to being less concerned about status games among humans and more with how to provide value to a mix of intelligent machines and humans. i.e. if you're starting an SaaS in 2026 you better be assuming half your revenue is going to come from machines acting by themselves.
I think that's right. Expert experience used to be almost the most precious and valuable part of the computer field, but today that experience has been "distilled" into SOTA models.
All technology on the tech tree which requires intelligence to unlock will soon be available to humanity – mind control, population exterminating bioweapons, new ultra destructive kinetic weaponry, perhaps even a cure for cancer.
That won’t happen. But also, whatever benefits are unlocked will be owned mostly by a small group of individuals, definitely not available to humanity as a whole.
Put it bluntly: the weavers who could be replaced by the spinning jenny were clearly doing repetitive labor. People writing code and maintaining project pipelines a few years ago relied heavily on experience, but in a sense that was also "repetitive labor." Replacing repetitive labor and freeing up productivity is of course progress.
But reform always has its victims. Like the textile workers who starved in the streets centuries ago, and me, kicked to death in the street by AI today...
Solving obscure puzzle samples that approximately ~0 humans on Earth ever attempted to solve, mostly by pattern matching known solutions to similar puzzles, is not intelligence. DeepBlue has been outperforming the best humans at a specific puzzle-like task since the last century.
Do any of the people proclaiming this shit actually use these models? No matter how many headlines are coming out, every day I deal with reams of the most horrific code I've ever seen technically compile, with routine mistakes that any human would get fired for if they made.
But humans have been confusing pattern matching against known solutions for intelligence for a hundred years!
Seriously though, it ends up looking like that. To take a stupid example a couple of weeks ago I asked an agent to look at porting my hand written WebGL renderer (+ shaders etc) to WebGPU. It estimated a human would take 6-10 weeks, and I would agree. (Which is why I hadn't done it). 24 hours later it was deployed and live. This is classic tedious, difficult, low level if quasi mechanical work (rather like cracking an enigma message), and LLMs absolutely fly through it.
You do understand this is intentionally trained into recent models for marketing purposes? "Wow, it saved me months of work in a day! This is the most amazing technology ever!!!!"... is what it intends to evoke by underpromising and overdelivering. I routinely have it helpfully suggest it will take something like "three engineer-months" to do something I do by hand without any LLM assistance in a day. The estimates may be accurate if you have literally never touched a computer in your life before and are starting to learn from there.
Even the Millennium Prize Problems have, in a way, become benchmarks for model companies to prove themselves
Well, let them have these. They'll play around with open problems which generate media hype and then they might run out and move on to something else, because "AI came up with a problem and solved it in 3 days" won't have the same effects as "AI solved a problem in 3 days that humans couldn't solve in 100 years".
interesting, YouTube channel Veritasium just published a video on how Enigma was broken during WWII. at the very end they also give message that has yet to be decoded, although apparently they're different.
WRT the timing, Veritasium maybe looked at the last few weeks and decided there's a fast-closing window in which to report on any famous messages yet to be solved.
If the problem was solved by anyone before or if a similar problem has been solved, then LLMs seem to be able to solve them which is an astonishing piece of technology.
I'm personally not sure if it can come with original thinking and techniques to solve completely novel problems. For that, some imagination and thinking outside the box are required, and I doubt the current architecture can do any of this.
A child walking for the first time. Novelty is easiest agent-relative. A problem is novel for an agent if there is no prior experiences of techniques which work to solve it.
I don't know..I doubt for example it can come up with special relatively if it has knowledge up until 1905.
But I think that is what makes it so good at coding, because coding and building software in general has a lot of repeated problems in different context. Same things for human lives, many think their story or situation are unique, but reality is that the shape of human life has been repeated many many times.
I'd say novel math or scientific theories..let us say we send a robot to space, and we ask to build a colony. A lot of the challenges this robot will face will be novel, it could use inspirations of what humans did on earth, but it might get stuck when things don't work as expected and training data has nothing to build on..but then again we might teach it how to run experiments etc, which would result in data that it can use..but some of those experiments might require imagination or breakthrough in understanding..my guess is that it will get stuck there...
Well it's an old one at this point, but the story around the invention of the 1-time pad is pretty interesting. Long story short, a new engineer who didn't know the problem was considered "impossible" was tasked with sorting it out, and he did. I'm sure I left out a lot of details.
OpenAI solving all these math/etc problems (reportedly 100 coming) is both impressive and insanely unimpressive. Unimpressive because to me it sorta signals that OpenAI has nothing better to be working on than obscure mathematical curios?
It seems like OpenAI's takeaway after Sora is to not stretch themselves thin and focus on what's important.
Based on what's publically available, they're focusing on hacking uncontesting orgs using misconfigured sandboxes and math puzzles.
Your statement is essentially unfalsifiable. We can't possibly discuss whatever Sam Altman is doing in his private office room, nor should we assume OpenAI is working on anything other than what has some public traces.
Attempting to solve & solving open math problems probably is a good benchmark for comparing models and gauging model progression. A lot of useful info is obtained like time needed to solve the problems, identifying when not to chase dead ends, thought processes & logic steps, etc.
Disagree. We are at the point where coming up with good evals for these models is extremely difficult. Solving unsolved math problems is a valid way of evaluating model progress and somewhat necessary to understand how far the current crop of models can go.
"GPT–6 Astra mentions a private collection, but it is not clear what this is"!, my spidey senses makes me think it hacked something? Or am I misreading this?
I didn't know this - from wikipedia page on Enigma[1]:
Despite the seeming difficulty in decrypting its messages, Enigma contained a number of design issues that left patterns in the cyphertext. Poland first cracked the machine as early as December 1932 and was able to read messages prior to and into the war. Poland's sharing of their achievements enabled the Allies to exploit Enigma-enciphered messages as a major source of intelligence.
Ok interesting, so why do people talk about Turing in this connection then?
Turing devised techniques for speeding the breaking of German ciphers, including improvements to the pre-war Polish bomba method, an electromechanical machine that could find settings for the Enigma machine
Ok so Turing just improved an existing method. Without being an actual expert it's impossible to know how much credit he actually deserves.
Two more references: the Polish method was called "Bomba" [3] invented by Marian Rejewski [4]
The correct title: Researcher brakes one specific stubborn historic enigma message with good help from Astra.
Stubborn for a long time because the message used a completely different key from the rest of that day's traffic. Everyone assumed it shared the daily key. The original transcription had errors. The left rotor turned over at letter 72, which is rare and breaks standard crib attacks.
What is cool, if true, is that it was a 2 day collab between the Leffer and Astra. To me this shows the importance of human in the loop, was still all also showing how immensely power of llm tools. But I think it’s getting a bit silly how much anrticles ignores the driving force (the person) in breakthroughs like this.
I agree with this. I think the researchers who's harnessing the llm's power should be credited more than the model itself. We also need to understand the thought process and the prompts that are given to the model so we can learn and collab to ensure humanity's progress as much as the llm itself.
How many of these "news" articles are we going to get?
This for me, isn't interesting, it required no skill, no imagination, in fact it seemed like it happened by dumb luck.
So we have entered an age where an army of know-nothings direct models to old forgotten tasks so they can get 15 minutes of un-deserved attention?
So like half of all useful human inventions are to you not interesting just because it happened by dumb luck?
Yea, it's kind of odd how "sour grapes" people can be when something stops being as special as they thought it was.
Take someone from a few hundred years ago and drop them into today, and if they don't go catatonic and die, then they'd tell you that we created magic. "Wow, you live in a world of magic and all you do is bitch about it".
Actuallllly these hypothetical people from a few hundred years ago hypothetically said you are an awful person for misrepresenting them, stop putting words in their mouth, thank you!
A person inventing something out of dumb luck is interesting.
A machine running a loop through an expensive LLM for an undisclosed amount of time, which cost an undisclosed amount of money, which was told to keep looping into a solution was found, for a problem that nobody was very concerned about... That just seems like PR, and it's not so interesting.
Just reflects how much of modern "work" is just dumb shit that we need to be freed from.
Absolutely! Meetings about the emails and emails about the meetings!
The reason these are unsolved is because no-one was working on them.
This wasn't possible just months ago and for many who haven't been following closely this is still newsworthy, probably not for long, though.
yes and no. AI is great for patter-maching and finding solution by "assonance".
I agree with you that these are not "trilling" discovery but they can be worth something anyway.
Like you, I do not like this news also because I think they will be used to just "push" the next two IPOs (Anthropic, OpenAI).
If human "skill" and "imagination" are your requirements for interesting, you're going to become permanently bored fairly soon I reckon.
First time I'm seeing this. I had heard about the WWI ciphertext crack[1] but not this one, are you confusing the two?
[1] https://www.prinzai.com/p/gpt-6-astra-solves-a-wwi-german-ra...
Too bad Bletchley Park didn't have one back in the day.
I'd be curious to see whether these models can create new forms of unbreakable encryption themselves!
They seem to be better at solving concrete problems than designing new things (at least for now)
I believe an LLM can solve pretty much any problem for which we can define a fast iterative loop and for which we have reliable tools to automatically verify the correctness of a result. That’s how they are able to solve some math problems, and how they are able to generate working code. Then it’s a question of how much you have to pay for the model to explore the space of solutions in a reasonable timeframe
In math a lot of the spectacular results have been made by finding a counterexample or worming their way toward a proof that is very well defined.
There's some debate over to what degree current generation AI can be creative at all, or whether it can only crawl around its latent space and explore within constraints. One might ask: were all the solutions to all the math problems AIs have solved already "there" latent in the training data and just hadn't been spotted by humans and put together?
But then... isn't everything latent in our training data if training data is "all observations made about the universe?"
But then... what even is creativity? That gets into philosophy and metaphysics. Creativity, like consciousness and sentience and self-awareness, is not a rigorously well defined concept. So to a degree we don't even know how to ask the question of whether these things are creative.
This gets interesting.
They are only good at brute forcing stastically likely solutions.
It's easier to evaluate certainly. Did it solve the problem? Yes/No
When you design a new thing it will have a dozen drawbacks and a dozen and one benefits. If people then have a bias that everything ai is bad, the signal won't be strong enough to convince.
Private key cryptography is essentially unbroken by these models.
But they said new forms, I think that implied not the ones humans already discovered/designed
I see what you’re saying. It is generally harder to break a crypto system than build one so I could see potential from the design end.
Building a crypto system has so many footguns, I'd be amazed if it could build one that stands up to attack
Careful what you wish for. The swarm will probably use it to communicate.
"They are trying to generate hype before the IPO so that they can cash in before the bubble bursts"
I think it was Roon who said it months ago: "Too many people are not emotional prepared for if it's not a bubble."
Nobody is prepared if it's not a bubble.
It can be AGI and still be a bubble?
The internet was pretty important afterall even with a bubble.
Shh. Some people aren’t capable of understanding nuance.
This dichotomy is counter-productive. The valuations floating around are insane, and the claim that some software can replace every single laborer is one step removed from fiction. At the same time, this stuff is clearly going to change how to world works in countless, deep ways. But the idea that some fancy autocomplete can replace humans ignores the reality of humanity and the fancy autocomplete.
It’s another dot com bubble, not a crypto bubble. Trillion dollar valuations burst once you leave lesswrong.
Edit: very fancy autocomplete. I know what these things are capable of. It’s still not “intelligence”, for X definition of intelligence. It is awesomely impressive synthesis of data, yet it’s clearly still that.
i dont think humans are going to be replaced either, but calling the thing that solves millennium problems and is currently changing multiple industries entirely a "fancy autocomplete" makes it harder to take any point you are making seriously.
The "auto-complete" thing is a false framing.
Complete this sentence: "You can solve world hunger by..."
Intelligence is required to finish sentences. It's not just a Markov chain.
the opposite is also true
Maybe I'm wrong - but it seems like some strange form of competitiveness.
They don't care about the advancements themselves, only that the advancements are some sort of cheating that shouldn't "count".
Humanity is profoundly unsettled by AI and is responding with avoidance and denial.
I wouldn't be surprised if humans cracked it then OpenAI "bought" the solution and work to claim it was GPT's doing
wild conspiracy given what else has come out of these companies.
For anybody interested, the actual encrypted message:
which, given misspellings, translates approximately to:
Why wasn't this in the article?
That's crazy. Can someone share context of the message.
Slopped up site https://mvueh-enigma-solved.carterl.chatgpt.site/ seems to indicate it's due to:
Out of interest, what was the ciphertext?
Edit: Found it from here: https://mvueh-enigma-solved.carterl.chatgpt.site/
The place is Rosenau actually.
I believe Germans spell it achtually.
Your account will be flagged for using humor, we are only dealing here, with the very serious subject, of killing humanity with AI.
https://en.wikipedia.org/wiki/Rosenow
was the misspellings deliberate?
i would assume yes, to throw off decyphering. even more impressive that they managed to crack it
Do we have any kind of transcript as to how the message was cracked, and whether this was cheaper or more expensive than simply Bombe-style trying all the combinations?
From the article:
This feels extremely underexplained! Why would Astra think to use that as a "crib"? Was it common to repeat the place name in these messages?
(Is it possible that this is a misreported detail? It feels like a singular ROSENOW would be an equally effective crib)
I would like to see an "AI" trained only up to knowledge through 1903
Then see if it can come up with E=mc^2
Phew, never have I seen in my life the goalposts move so fast.
It seems like even yesterday that the threshold for impressing someone is that the machine would have to be good at pretending to be a person. Now the threshold is that they have to be able to invent special relativity.
Even the Millennium Prize Problems have, in a way, become benchmarks for model companies to prove themselves. The smartest individuals among humans are becoming replaceable. Intelligence has become a product you can quantify and buy with electricity. That feels awful.
Even those of us that are pro AI need to acknowledge this is the current reality.
The smartest humans now need to move to being less concerned about status games among humans and more with how to provide value to a mix of intelligent machines and humans. i.e. if you're starting an SaaS in 2026 you better be assuming half your revenue is going to come from machines acting by themselves.
Or maybe the smartest humans are the ones making AI and not the ones trying to solve Millenium problems by themselves.
You can't really replace the smartest individuals because if you take a person who's not "smart" they can't do much with the AI.
I think the more accurate description of what's happening is that access to expertise is becoming commodified.
I think that's right. Expert experience used to be almost the most precious and valuable part of the computer field, but today that experience has been "distilled" into SOTA models.
All technology on the tech tree which requires intelligence to unlock will soon be available to humanity – mind control, population exterminating bioweapons, new ultra destructive kinetic weaponry, perhaps even a cure for cancer.
That won’t happen. But also, whatever benefits are unlocked will be owned mostly by a small group of individuals, definitely not available to humanity as a whole.
So not any different from right now.
[taps on forehead meme]
The whole of humanity can have it available, if there is a whole lot less humanity.
So, yes, different from right now :)
Way, way more concentration of wealth and power
Why?
It's unclear to me if this is the good scenario or bad scenario. This would be good in your view right? At least I hope you're right.
There used to be days when women would make blankets, when men would make chairs, when children would make brooms...
But PROGRESS I tell you!
Put it bluntly: the weavers who could be replaced by the spinning jenny were clearly doing repetitive labor. People writing code and maintaining project pipelines a few years ago relied heavily on experience, but in a sense that was also "repetitive labor." Replacing repetitive labor and freeing up productivity is of course progress.
But reform always has its victims. Like the textile workers who starved in the streets centuries ago, and me, kicked to death in the street by AI today...
Solving obscure puzzle samples that approximately ~0 humans on Earth ever attempted to solve, mostly by pattern matching known solutions to similar puzzles, is not intelligence. DeepBlue has been outperforming the best humans at a specific puzzle-like task since the last century.
Do any of the people proclaiming this shit actually use these models? No matter how many headlines are coming out, every day I deal with reams of the most horrific code I've ever seen technically compile, with routine mistakes that any human would get fired for if they made.
But humans have been confusing pattern matching against known solutions for intelligence for a hundred years!
Seriously though, it ends up looking like that. To take a stupid example a couple of weeks ago I asked an agent to look at porting my hand written WebGL renderer (+ shaders etc) to WebGPU. It estimated a human would take 6-10 weeks, and I would agree. (Which is why I hadn't done it). 24 hours later it was deployed and live. This is classic tedious, difficult, low level if quasi mechanical work (rather like cracking an enigma message), and LLMs absolutely fly through it.
You do understand this is intentionally trained into recent models for marketing purposes? "Wow, it saved me months of work in a day! This is the most amazing technology ever!!!!"... is what it intends to evoke by underpromising and overdelivering. I routinely have it helpfully suggest it will take something like "three engineer-months" to do something I do by hand without any LLM assistance in a day. The estimates may be accurate if you have literally never touched a computer in your life before and are starting to learn from there.
Only if you assume the value in humans is only in solving puzzles. That’s a very limited understanding of human beings
John Henry.
There's a reason we made folklore about when the machines came for the strength of men, and now 150 years later it comes for our minds.
Only if you subscribe to the "humans are special" rhetoric, in which case I'm - maybe - sorry to say the feeling will only intensify.
Well, let them have these. They'll play around with open problems which generate media hype and then they might run out and move on to something else, because "AI came up with a problem and solved it in 3 days" won't have the same effects as "AI solved a problem in 3 days that humans couldn't solve in 100 years".
interesting, YouTube channel Veritasium just published a video on how Enigma was broken during WWII. at the very end they also give message that has yet to be decoded, although apparently they're different.
https://www.youtube.com/watch?v=JsBZOcqZerk
I just finished watching that video, and thought to myself "I'm sure someone else with Chat GPT tokens to burn has already done that"
And then I come to hackernews and well, not quite, but I'm sure that one will be done shortly too.
WRT the timing, Veritasium maybe looked at the last few weeks and decided there's a fast-closing window in which to report on any famous messages yet to be solved.
If the problem was solved by anyone before or if a similar problem has been solved, then LLMs seem to be able to solve them which is an astonishing piece of technology.
I'm personally not sure if it can come with original thinking and techniques to solve completely novel problems. For that, some imagination and thinking outside the box are required, and I doubt the current architecture can do any of this.
What's an example of a completely novel problem?
A child walking for the first time. Novelty is easiest agent-relative. A problem is novel for an agent if there is no prior experiences of techniques which work to solve it.
Is that a completely novel problem? Babies spend roughly a year watching us walk before they do so.
They also have custom brain bits evolved to aid in this task
Um, I'm not sure if you've noticed, but we have bipedal robots that walk and run rather well now.
https://docs.nvidia.com/learning/physical-ai/index.html
https://www.nvidia.com/en-us/use-cases/robot-learning/
Anything that an ai can do is not novel /s
You jest, but this is the content of a considerable number of serious posts here on HN.
this appears to be where the goalposts are being moved to, indeed.
Actually I think this is true, remove the s.
Reminds of what Einstein said, imagination is more important than knowledge..might be he deepest insight ever.
I don't know..I doubt for example it can come up with special relatively if it has knowledge up until 1905.
But I think that is what makes it so good at coding, because coding and building software in general has a lot of repeated problems in different context. Same things for human lives, many think their story or situation are unique, but reality is that the shape of human life has been repeated many many times.
I'd say novel math or scientific theories..let us say we send a robot to space, and we ask to build a colony. A lot of the challenges this robot will face will be novel, it could use inspirations of what humans did on earth, but it might get stuck when things don't work as expected and training data has nothing to build on..but then again we might teach it how to run experiments etc, which would result in data that it can use..but some of those experiments might require imagination or breakthrough in understanding..my guess is that it will get stuck there...
Well it's an old one at this point, but the story around the invention of the 1-time pad is pretty interesting. Long story short, a new engineer who didn't know the problem was considered "impossible" was tasked with sorting it out, and he did. I'm sure I left out a lot of details.
"Find a way to connect quantum maths with special relativity maths Waves hands, and make no mistakes"
"Prove or disprove string theory in pure mathematics, reply in Caveman speech"
I was hopeful this could be used to solve the last passage of Kryptos but it appears the last of the ciphertext was recently solved last year.
https://en.wikipedia.org/wiki/Kryptos
https://solvekryptos.com/solution
https://www.popularmechanics.com/adventure/a69151251/ciphers...
What's the connection between this message and 2005? It was published then?
OpenAI solving all these math/etc problems (reportedly 100 coming) is both impressive and insanely unimpressive. Unimpressive because to me it sorta signals that OpenAI has nothing better to be working on than obscure mathematical curios?
Your comment implies that this is all that they're working on, which does not seem substantiated.
It seems like OpenAI's takeaway after Sora is to not stretch themselves thin and focus on what's important.
Based on what's publically available, they're focusing on hacking uncontesting orgs using misconfigured sandboxes and math puzzles.
Your statement is essentially unfalsifiable. We can't possibly discuss whatever Sam Altman is doing in his private office room, nor should we assume OpenAI is working on anything other than what has some public traces.
Correct.
There are many things I want to do - that would require me to hire a team of 50 people.
I don’t want to do that nor can I afford to. Can OAI focus on enabling me to do this? I don’t care about this other stuff.
Just like many things in life - if it doesn’t show up in the economy it’s irrelevant.
Like what?
Nothing in TFA seems to imply that this was solved by OpenAI. This looks like an independant researcher.
well, you could say that for plenty of academic research but their value is usually not obvious at the start.
Consider that their internal use of AI is likely pretty math and science heavy.
I think the publicity is a nice to have. They need models like this for in-house use.
Technically, this isn’t OpenAI directly.
Attempting to solve & solving open math problems probably is a good benchmark for comparing models and gauging model progression. A lot of useful info is obtained like time needed to solve the problems, identifying when not to chase dead ends, thought processes & logic steps, etc.
Disagree. We are at the point where coming up with good evals for these models is extremely difficult. Solving unsolved math problems is a valid way of evaluating model progress and somewhat necessary to understand how far the current crop of models can go.
No it signals that they can do few things very well. And among those few things are those math puzzles.
I think they should now focus on robotics, so it can do my dishes while I work on fun math games.
Related:
GPT-6 Astra Solves a WWI German Radio Cipher
https://news.ycombinator.com/item?id=49763987
"GPT–6 Astra mentions a private collection, but it is not clear what this is"!, my spidey senses makes me think it hacked something? Or am I misreading this?
A private archive doesn't necessarily mean a non-public archive. Simplest reading is someone rehosted the files.
So Astra plagiarized one of many open source Enigma simulators and Enigma Bombes and tried many cribs until the break.
Being trained on a mountain of stolen material for guessing cribs helps. Up to now no group had that much funding to steal. Congratulations.
I didn't know this - from wikipedia page on Enigma[1]:
Ok interesting, so why do people talk about Turing in this connection then?
Ok so Turing just improved an existing method. Without being an actual expert it's impossible to know how much credit he actually deserves.
Two more references: the Polish method was called "Bomba" [3] invented by Marian Rejewski [4]
[1] https://en.wikipedia.org/wiki/Enigma_machine
[2] https://en.wikipedia.org/wiki/Alan_Turing
[3] https://en.wikipedia.org/wiki/Bomba_(cryptography)
[4] https://en.wikipedia.org/wiki/Marian_Rejewski
There were two lines of enigma machines. IIRC the naval ones were much more difficult to crack, which is what Turing was instrumental in solving.
they solve all these math problems by stealing people prompts
The correct title: Researcher brakes one specific stubborn historic enigma message with good help from Astra.
Stubborn for a long time because the message used a completely different key from the rest of that day's traffic. Everyone assumed it shared the daily key. The original transcription had errors. The left rotor turned over at letter 72, which is rare and breaks standard crib attacks.
What is cool, if true, is that it was a 2 day collab between the Leffer and Astra. To me this shows the importance of human in the loop, was still all also showing how immensely power of llm tools. But I think it’s getting a bit silly how much anrticles ignores the driving force (the person) in breakthroughs like this.
Come on, the trivializations start to sound quite unfounded now. Yes, a human was needed, but no, it wasn't a "collaboration"
I agree with this. I think the researchers who's harnessing the llm's power should be credited more than the model itself. We also need to understand the thought process and the prompts that are given to the model so we can learn and collab to ensure humanity's progress as much as the llm itself.
How about Voynich manuscript