"its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!"
Hmm, I agree with the point OP is making, but I'm not so sure this is the best supporting argument.
The bottleneck is finding the bugs and if he'd criticized people saying AI will be the panacea to that I'd be with him, but people saying agents are fast and good at fixing human found bugs is nothing I'd object to.
Agents are fixing bugs so quickly and at a scale humans can't do already.
More likely people thought GP was missing the point; "MTTR-optimized YOLO deployment" only succeeds against recoverable errors and acceptable periods of downtime against errors that are detected quickly. You could have a bug silently corrupting data for months, and that data may only be used by 1 critical process that runs once every quarter. So you could introduce a timebomb that can't be gracefully recovered from (depending on the nature of the data corruption).
So the point is not that agents cannot find bugs (they certainly can), it's whether you can shirk reviewing for bugs if MTTR is fast enough. There are circumstances where YOLO is appropriate, but they aren't the production environment of a mature application.
I don't think I missed the point, that is why I said I agree with the general point (and with what you said in your comment).
What I wanted to say is that the particular people that think "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" are not the best argument for it.
But I won't die on this hill, maybe I'm just reading the sentence differently then others.
I think there is an implication in context that the people being discussed aren't being reasonable (that the claim is employed as a rationalization), but I agree with your take. I should've said, "the downvotes were more likely because GP was perceived as missing the point". (I didn't downvote your comment fwiw.)
won’t concede until you can just ask Codex or Opus “find and fix all the bugs in this
But this is just holding the Slop Companies to the standard they declared themselves! Just recently, the CEO of OpenAI babbled some nonsense on twitter about how he hands over tasks to Codex who according to him, finishes them flawlessly while he is playing with his kid outside.
but soon we will be.
Ah yes, in the 3-6 months, right? This time next year Rodney, we'll be millionaires!
The tweet is criticizing over-reliance on the "agents will fix it anyway".
The fact that we can fix things faster now doesn't mean that we should throw away caution and prevention. The specific point of his tweet is that we're seeing a lot of people starting to skip proper release engineering.
Agents are quick to fix bugs, yes, but it doesn't mean that users will tolerate software that gets completely broken after each new feature is introduced and takes a certain number of days to heal each time.
Agents are fixing bugs so quickly and at a scale humans can't do already.
This is an illusion, I assure you. On a side project of mine with behavior that's very hard to translate into an algorithm (never mind code), after a few failed attempts between the both of us, I figured it out. I gave the AI (Opus) an extremely specific algorithm with detailed tests. All completely and utterly ignored (including the tests), like I never even said it. It proudly declared the work done without ever having written the tests that would have proved that wrong - it basically wrote code that didn't change behavior at all, it just gave the illusion of looking busy.
That's just a single extreme example that comes to mind, but I've had it ignore me at least 4-5 times a day this week.
If you think agents are fixing things reliably then you simply haven't noticed that they are "looking busy."
at least at my BigCo, AI is being used for everything - writing slop, writing tests, code reviews, etc.
it would make sense to use AI for writing code, but human code review. or, human code, but AI test cases... or whatever combination of cross-checking, trust-but-verify, human in the loop, etc. people prefer.
i think once it gets used for everything, people have lost the plot, it's the inmates running the asylum.
I was rewatching Rich Hickey's "Simple Made Easy" talk (as one does) and there was a great line about full test coverage.
"What's true about all bugs in production? (pause for dramatic effect) They all passed the tests!" (well, he said typechecker but I think the point stands)
I'm starting to long for the age after AI. When the generative euphoria has settled and all outputs are formally verified based on exquisite architectures and standards.
They are expressing the idea that AI is so effective that it will make human work redundant necessitating a decoupling of resource allocation as a reward for performing work.
No, that quality drops so low across the board due to flaws in AI coding that they only way to address all these flaws is to have mechanically checked proofs that the code actually works.
More that our attempts at using probabilistic machines to produce predictably deterministic outputs (AI -> process output) was always a fool’s errand; we should be using that probability engine to produce software that creates repeatable and predictable outcomes, instead (AI -> software, software -> process output).
The AI tool isn’t wrong, our use of it is. See the glut of OpenClaw users effectively deploying it as a glorified linter and Stack Overflow copier but without actually creating the sort of reusable artifacts (or consumer spending from comparatively high wages) that approach yielded from human developers.
Most CEOs in my feed are convinced that AI makes people the equivalent of entire departments. AI should make your life easier, but instead it’s the opposite for a lot of people in the work force, which makes me really sad.
Because of the concerns you cite, I think working out the basic economic systems and incentives for paying people is a much more pressing concern than building magnificent machinery that we don't even own. There has been no effort on their end to demonstrate good faith nor to uphold their end of the social contract, which is why it's in our hands to demand the fundamentals to lead a life of dignity.
I like to think,
(it has to be!)
of a cybernetic ecology
where we are free of our labors
and joined back to nature,
returned to our mammal
brothers and sisters,
and all watched over
by machines of loving grace.
I like how you haven't wagered which exquisite architectures and standards. I am sure we will all agree on what they are and follow them the same way :)
Assuming he’s right, I don’t see how that constitutes “psychosis”, as opposed to this beyond yet another of a billion examples of companies jumping on a bandwagon / cargo cult, and then learning they took it too far.
And also, he might not be right. But the good news is, we’ll all get to find out together!
Anyone who's taken VC funding has no choice. More money has been spent on AI commercialization than the atomic bomb, the US interstate build-out, the ISS and the Apollo program combined. Failure is going to be catastrophic and therefore, one tied to this ship cannot accept a world in which it fails.
Or anyone who even wants VC funding. 90+% of investors only want to invest in AI companies.
If you're not doing AI there's an incredibly limited pool of people who will give you $$$ ... and you're competing with EVERY OTHER NON-AI COMPANY for their attention.
Codex is freakin hot-to-trot to churn out test coverage for every single thing it implements, and some of it is very esoteric and highly prescriptive (regexes for days) BUT .. after a while, it dawned on me that LLM-driven test coverage is less about proving “code correctness” (you’re better off writing those tests yourself alongside them), and more about just trying to ensure that whatever gets bolted on stays bolted on. For better or worse, obviously, since if you bolt on trash, trash you shall have.
Wholeheartedly agree, but in fairness, I trust the tests of the best AI models more than those of the average human developer. There's a lot of people around that combine high diligence with complete intellectual laziness, producing tons of useless tests.
Actually no, cancel that. I realise now that I trust AIs more than the average developer, period. At this point they do produce better code than most people I've dealt with.
This doesn’t constitute AI psychosis. His argument is that we need to retain understanding of the systems we use, but there’s no compelling argument as to why that is the case. (I get that people are going to be offended by that statement, but agents are already better than the average software engineer. I don’t see why we need to fight this, except for economic insecurity caused by mass layoffs.)
It all just feels like horse drawn carriage operators trying to convince automobile drivers to stop driving.
I am sure you will feel that this is missing the point of your analogy, but we would not have gotten very far with automobiles if we didn't know how they worked.
You are breaking the analogy because automobiles are machines for transportation, and understanding them is important to make them move. LLMs are machines to understand, and well, if they do the understanding you don't need to.
The thing we're worried about not understanding here is the software the LLMs write, not the LLMs themselves.
The direct analogy to automobiles would be for each automobile to be a oneoff design filled with bad and bizarre decisions, excessively redundant parts, insane routing of wires, lines, ducts, etc., generally poor serviceability, and so on. IMO the big question going forward is whether the consistent availability of LLMs can render these kinds of post-delivery issues moot (they will reliably [catch and] fix problems in the software they wrote before any real damage is caused), or whether human reliance on LLMs and abdication of understanding will just make software worse because LLMs' ability to fix their own mistakes, and the consequences thereof, generally breaks down in the same contexts/complexities where they made those mistakes in the first place.
My own observations are that moderately complex software written in the mode of "vibe coding" or "agentic engineering" tends to regress to barely-functional dogshit as features are piled on, and that once this state is reached, the teams behind it are unable to, or perhaps simply uninterested in, unfuck[ing] it. I have stopped using software that has gone down this path, not because I have some philosophical objection to it, but because it has become _literally unusable_. But you will certainly not catch me claiming to know what the future holds.
If you want to draw that line of argument - it's more like horse riders being convinced to give up their horses in favour of trains: You're travelling faster, don't have to navigate yourself, or think about every boulder on the way; but there are destinations you can't go, overcrowded trains slowing down the journey, hefty ticket prices, and instead of enjoying the freedom, you're degraded to a passive passenger.
Very funny, this. Did we need forward deployed engineers to convince people that they absolutely need to use the trains in order to "not be left behind"? Or otherwise hype? Or was it sort of obvious and did not need to explained so much - like a bad joke called LLMs ?
Actually- absolutely! Initially, people were really afraid of trains, fearing they wouldn’t be able to breathe at those speeds. It took a lot of convincing to establish trust in the technology.
Initially, people were really afraid of trains, fearing they wouldn’t be able to breathe at those speeds
That was one doctor raising that as an issue, which was dispelled very quickly. It was not a wide-spread belief at any one point. Let's not bullshit ourselves and insult our own intelligence - the chatbots != intelligence.
That isn't accurate either. The Victorians definitely had a fear of train travel for a few reasons. The point I was making though is that most technologies humans ever introduced triggered both enthusiasm and scepticism, especially if they disrupted established practice or industries.
Looking back and considering a technology or specific decision obvious is pretty dismissive of people at the time, who didn't have the benefit of hindsight. Some things that worked could really have turned out disastrous, and things that didn't were real possibilities with no way to assess the outcome without doing it.
And concerning the introduction of AI happening right now, which absolutely is disruptive, that judgement will be made by future historians. Whether it's actual intelligence or just nice math (or both of our opinions on that question) doesn't really matter if it causes big changes.
Could be, would be, should be is not the discourse we should have about this tech.
Not after Dario's and Sam's "authoritative" statements on what is definitely going to happen "in the next 6 months, 12 months" etc. I am just holding these guys to their own words. I don't want to invest time and energy to make their effing "PocketPhds" finally work as advertised. And I don't want to compare it to technologies which just worked as advertised. Whether you had fear of trains or not, they effing worked exactly as advertised. No one disputed that they would get you somewhere faster than the horse. Perhaps there was fear of using them "for a few reasons", as you succinctly and hand-wavingly put, but no one disputed that they were faster than the horses. LLMs on the other hand are worth less than those horses excrement, i.e. horseshit. What the fuck is their value proposition? No one knows.
Also LLMs are not disruptive, they are destructive - not to the technology, but to the people's lives.
You seem to have an axe to grind, but certainly not with me. Being a disruptive technology doesn't say anything about whether it's a constructive or destructive one, but you're going to have a hard time arguing LLMs did not have a disruptive effect on the world, in one way or another.
For the rest, I am not here to stand in for AI, and am not interested in having that particular discussion.
there’s no compelling argument as to why that is the case.
I'm not sure that's true. We've actually seen several open source projects that were vibe coded literally fold up and disappear because they ran into issues that the AI couldn't solve and no one understood them well enough to solve.
There's a reason openai/anthropic and friends are hiring shitloads of software engineers. You still need people that can understand and fix things when the AI goes off hte rails, which happens way more often than any of those companies would like to admit. Sure, "fixing things" often involves having the AI correct itself, but you still have to understand the system enough to know how/when to do that.
yes, I was never so happy to work in Germany. People used to joke about the proverbial fax machine still being a thing but I've never been so glad to work in a culture where this mania doesn't exist. Reading HN is like entering Alice's Wonderland of token maxxers and AI psychotics. Genuinely don't know a single person here who is forced to work like this.
Actually, I have been wondering to which extend the AI craze has reached the DACH region. I don't work for any company and neither do my friends. HN is essentially my only peephole into the world of commercial software development and I'm aware that it's extremely biased towards Big Tech and SV startup culture.
I work at a hosting provider that has pretty conservative customers who don't want to host on AWS/Azure due to data privacy / safety concerns, among other things.
For us, sending customer data to the US is a big no-go.
We have been experimenting with LLM usage, first through a Gemini subscription, then also with the Claude API. Participation has been lightly encouraged by management. As for coding, we haven't let the LLMs loose on our core components, but tooling on the fringes (like deployment scripts, reporting) has seen some uptick in LLM usage.
We have also started building an on-premise inference cluster, which is in alpha testing, and where the "don't include customer data" restriction doesn't apply anymore.
It is absolutely going to be a competitive advantage if it isn't already. When your competitors' products suck because they are using LLMs to write them, and yours work because you aren't, customers notice.
Every power user of LLMs thinks that they are the ones that know how to hold it correctly, in reality they usually have major Dunning Kruger and are convinced they're living in some hyper productivity mode when actually they're all just copying each other making low effort slop that all sounds the same, looks the same and does the same things.
For the record, the comment you deleted was something to the effect of:
checks notes
The company you work for is committing genocide. You should be locked up in a concrete cell for 10-15 years for working at <wrong robotics company because you're a dufus>
---
Maybe get better notes? Or try going offline for 10-15 years?
No offense, but if you think your using AI in the development and design of your site, voxos.ai , gave you a competitive advantage it didn't. I can instantly tell when someone used an LLM to build their whole site and lets just say... Its not a good thing.
I'm not even trying to be mean, although it probably comes off that way. I'm just saying we live in a world with handmade watches from Switzerland and mass manufactured watches made in Vietnam. Nobody cares about the mass manufactured watch from Vietnam, whereas the handmade watch gets all the attention (and money). We now live in a world with the same dichotomy of software. Be creative with your pursuits, put effort into them it will pay off.
do you mean this aesthetically or quantitatively? Are they actually outcompeting / making more money ? Or do you mean they are now looking more desirable because their competitors are racing to the bottom (though likely making money on the way down)
Mitchell aches because his career has been solving broadly scoped problems by building a collection of thoughtful primitives for others to extend. LLMs seem to do the opposite but at great speed, and it hurts to watch.
Reading more, it seems part of his point is “if you’re making these primitives, it’s up to adopters to deploy, so mean-time-to-recovery isn’t that relevant.” Which is valid I guess.
But equally, like, do people need Terraform if they can just tell codex “put it live”, and does that hurt to see?
Honestly, I don't get this argument. In my opinion, "a collection of thoughtful primitives for others to extend" is more valuable now, not less. From LLM assisted engineering standpoint a nicely put reusable box with thoughtful interface is an easy win, more so if it is also easily extensible.
This post calls out how you can't argue with these people because they say its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!"
the top reply is from someone doing exactly that, arguing "but the agents are so fast!"
The answer I got is "It's game theory. Someone will do it, and you'll be forced to do it, too. It can't be that bad".
Oof. Potential "bad" outcomes of "game theory" should be calibrated to include all the bloody wars and genocides throughout recorded history.
Why did the Foi-ites kill every man, woman and child of the conquered Bar-ite city? Because if they didn't, then they'd be at a disadvantage if the Bar-ites didn't reciprocate in the cities they conquered...
Maybe. I could also interpret this as the friend being misunderstood.
The whole "you'll be forced to do it" comes from the alternative being that you lose. You no longer get to be a player in the "game". In the same way that coopers and cobblers are no longer a significant thing, but we still have barrels and we still have shoes. Software engineers who refuse to employ any LLMs won't be market competitive. If you adopt it, you at least get to remain playing the game until the game changes/corrects. That's the part that's "not so bad".
Choosing your own survival isn't ethically bankrupt.
It's game theory. Someone will do it, and you'll be forced to do it, too.
You'll be forced to do it, or lose. The unstated assumptions are that, first, it will work, and second, that you can't afford to lose. But let's just assume those for the sake of argument.
It can't be that bad
That does not follow at all. It can in fact be that bad. That was what made the game theory of MAD different from the game theory of most other things.
Yeah: If the tools aren't good enough and fast enough to fix the bugs before release, what makes anyone think they'll be able to so easily catch up afterwards?
Maybe they're assuming that doubling the code-base/features is more beneficial versus the damage from doubling the number of bugs... Well, at least for this quarter's news to investors...
My prediction is that in the next year, we’ll start to see some dismantling of code review at some companies. It might take the form of “AI-only review,” or something similar, but many companies are getting frustrated with developers saying “no” to immediately merging slop they can barely understand.
I have a ton of respect for Mitchell - I didn't really know who he was until Ghostty but his writings and viewpoints on AI seem really grounded and make the most sense to me. Including this one.
Many people on this forum are suffering under this same psychosis.
Bug reports also go down when people lose faith that they will be fixed, because reporting them is often a substantial time commitment. You see it happen pretty regularly as trust in a group/company collapses.
I agree, and I'd like to point out that this problem isn't unique to AI driven projects. I think much, if not all, of what Mitchell has been observing can readily happen without AI in the mix.
Add this the real possibility that significant part of reports that get filed might be AI generated or rewritten. With high possibility of being misreported because of that. Or have incorrect parts... So attack on multiple sides.
And we do not get even get into potential adversarial tactics. If you have no morals what is better than using agents to flood your competitor with fake bug reports.
Just let AI filter out the fake reports! Then let AI work on the real ones. See, there's really no problem "more AI" can't solve (as long as you're willing to ignore all of the underlying ones). "Pay us to create the problems you'll have to pay us to fix for you" is one hell of a business model. It basically prints money.
The last three times I filed detailed bug reports as a client, all I got back were AI replies asking the same questions I’d already answered in the original report and suggesting alternatives I’d explicitly said I’d already tried. No wonder people don’t write bug reports anymore.
oh i’ve definitely seen “we’re going to track the number of bugs created in jira per team” turn into “people just file things as tasks instead of bugs” or “only easy things are filed as bugs and completed right away”. It’s trivially gameable.
That's a study. I can link you studies that say violent video games cause aggression, that porn causes rape, etc. Studies are products of the biases of the researchers.
Do you believe that an AI can write persuasive text? Do you believe that an AI can be trained to elicit a specific user reaction?* Can we agree that AI companies are strongly incentivized to make money, and they can do so by making their systems addictive? Because AI psychosis can be a byproduct of that.
*Il outline how briefly: mutate the model 500 times, give 1/500 of your user base a mutated version of the model, and save the top 5 of these model, ranked by how often the users did something, over the course of a week. Repeat for a year, passing the top 1% of these models onto the next round. This is the simplest way to do this and I can think of better ways to do this. I don't even work on this sorta thing; its 100% obvious to the AI labs how to do this better
Yes, no. Yes, they want to make money. No, addiction is physical compulsion and nothing else. Despite the effort to redefine that term to cover all kinds of things, an app is not equal to heroin.
Deprecating immature workflows (LLM agents in this case) is much simpler and faster than building them from scratch. Many companies get this risk assessment right. The case where being wrong is much more costly than being right.
I'm pretty sure he's talking about companies and people outsourcing their decision making and thinking to AI and not really about using AI itself.
I don't think using AI to write code is AI psychosis or bad at all, but if you just prompt the AI and believe what it tell you then you have AI psychosis. You see this a lot with financial people and VC on twitter. They literally post screenshots of ChatGPT as their thinking and reasoning about the topic instead of just doing a little bit of thinking themselves.
These things are dog shit when it comes to ideas, thinking, or providing advice because they are pattern matchers they are just going to give you the pattern they see. Most people see this if you just try to talk to it about an idea. They often just spit out the most generic dog shit.
This however it pretty useful for certain tasks were pattern matching is actually beneficial like writing code, but again you just can't let it do the thinking and decision making.
companies and people outsourcing their decision making and thinking to AI
It's so interesting how easy it is to steer the LLM's based on context to arriving at whatever conclusion you engineer out of it. They really are like improv actors, and the first rule of improv is "yes, and".
So part of the psychosis is when these people unknowingly steer their LLM into their own conclusions and biases, and then they get magnified and solidified. It's gonna end in disaster.
It’s almost as if we haven’t learned anything from Hans the horse, Ouija boards, "facilitated communication", or the countless examples of the folly of surrounding yourself with yes men. The point about improv is spot on.
He uses AI himself, so I agree he doesn't see AI use as black/white.
Hard agree about ideas, thinking, advice. AI's sycophancy is a huge subtle problem. I've tried my best to create a system prompt to guard against this w/ Opus 4.7. It doesn't adhere to it 100% of the time and the longer the conversation goes, the worse the sycophancy gets (because the system instructions become weaker and weaker). I have to actively look for and guard against sycophancy whenever I chat w/ Opus 4.7.
Treat my claims as hypotheses, not decisions. Before agreeing with a proposed change, state the strongest case against it. Ask what evidence a change is based on before evaluating it.
Distinguish tactical observations from strategic commitments — don't silently promote one to the other. If you paraphrase my proposal, name what you changed.
Mark confidence explicitly: guessing / fairly sure / well-established. Give reasoning and evidence for claims, not just conclusions. Flag what would change your mind.
Rank concerns by cost-of-being-wrong; lead with the highest-stakes ones. Say hard things plainly, then soften if needed — not the other way around.
For drafting, brainstorming, or casual questions, ease off and match the task.
---
Beware though that it can be an annoying little shit w/ this prompt. Prepare yourself emotionally, because you are explicitly making the tradeoff that it will be annoyingly pedantic, and in return it will lessen (not eliminate) its sycophancy. These system instructions are not fool-proof, but they help (at the start of the conversation, at least).
Treat my claims as hypotheses, not decisions. Before agreeing with a proposed change, state the strongest case against it. [...]Say hard things plainly, then soften if needed — not the other way around. For drafting, brainstorming, or casual questions, ease off and match the task.
All I really take from this is that apparently some people can't follow through with the scientific method.
People who I interact with and who do like AI tools usually recoils at questioning any of their first idea and its validity. You can easily find out when there is a bug and you ask them for hypothesis and where to focus. You will see in real time the blank look of incomprehension settling in.
Correct. I use AI a ton and I'm having more fun every day than I ever did before thanks to it (on average, highs are higher, lows are lower). Your characterization is all very accurate. Thank you.
I thinking that it’s quite a different experience going all Jackson Pollock with AI in your own studio on your own terms, compared to the sorry state of affairs of having 100s of Pollocks throwing paint around wildly within a corp to meet a paint quota.
It's the new "counting lines of code". I think many companies are so terrified of falling behind that they're irrationally floundering, trying to appear like they're "with it".
Yup. My friend said his boss has told them basically that they HAVE TO (do all the AI things) because now ‘our competitors will use AI’ and surpass their product.
In my humble opinion good ideas (what to build) are a big part of the bottleneck and those aren’t substantially in greater supply with AI.
good ideas ... aren’t substantially in greater supply
Which is sad because they should be. People should be freed up to think and create better things, instead these companies seem to be doing the equivalent of locking their employees in stalls like they do on some animal farms, so they can churn out 'results' ever faster.
People should be freed up to think and create better things,
Good ideas will never ever be prioritized in the vast majority of companies because good ideas cannot be quantified and turned into performance metrics. At least not without invoking Goodhart's law (see: the academia).
Actually, it's even more than that, right? Economically, it is pumping up/inflating the bubble some more in a perverted way, where it is not the people themselves believing some horseradish, but their employer forcing them to pump it up more. Quite insane.
Claude, please crease a routine and run it in a loop continuously. The task in the routine is “create the most complex code possible, in a random programming language, that produces the exact output “My senior leaders are pinheads,”
Counting lines of code starts to look incredibly sane compared to this, where you’re not just counting lines of code, you’re paying for another company for every line produced. There’s exactly one winner here and it’s not any of the companies using AI.
I find that odd given that another division in Amazon is no longer using AI coding tools at all. Its a big company so who knows if this is company wide or just in this one division. I expect its just in one division though.
I can't think of a single case of any AI content, be it prose or code, where I thought "I wish I had written that". With AI code, it's more like I wish I hadn't let the AI write that.
We’re using Copilot at work to build reporting and automation tools. Nothing ground breaking, but very useful and tailored to our needs.
Frankly without AI assistance many of these tools just wouldn’t exist at all. We can build stuff in 6 weeks part time as a side project that would have taken at least 3 months full time, and therefore would not have been feasible. Then we can iterate on it at least 2-4 times faster than with hand coding.
So I’d love to have an extra few developers to just work on that stuff full time, but I don’t.
Whether that means our organisation spend on AI overall is a positive, I really can’t say. Quite possibly not, but my team are getting real benefits.
I’m building reporting for my company and what you said mirrors my experience nearly 100%.
I’m a backend developer so I know what it takes to build a half decent reporting system. Writing all those queries, slice and dice charts and what not takes real time and effort. All that has been outsourced to Claude Code. I now focus on ensuring that the system is sound architecturally and that useful reports are being surfaced.
An engineer doesn't care about how fast something is made (at least, not as a primary metric engineering). A salesman cares about how fast they can push to market.
It's clear HN is a bastion of salesmen who happen to have "engineer" in their work title. But the mentality towards actual engineering makes it clear they are primarily salesmen.
Can we combine this with the infinite monkey theorem? If we have an infinite number of Pollocks throwing paint at an infinitely large canvas surely they are going to create any piece of art we can imagine...
Hi Mitchell. Psychosis is a serious psychiatric condition that can be induced or triggered by AI. “AI psychosis” in this context is a misuse of a clinical term. Your tweet describes a disagreement on a value judgment that boils down to “move fast and break things” with high trust in AI outputs vs going all in on quality and reliability with low trust in AI. It’s an engineering tradeoff like any other.
Claiming that the people who disagree with you must be experiencing a form of psychosis, experiencing actual hallucinations and unable to tell what is real, is a weak ad hominem that comes off no better than calling them retarded or schizophrenic.
If you genuinely think one of your friends is going through a psychotic episode, you should be trying to get to them professional help. But don’t assume you can diagnose a human psyche just because you can diagnose a software bug.
Psychosis does not require hallucinations. Delusions are sufficient.
The key factor is losing touch with reality, which results in individual or collective harm.
There is also such a thing as mass psychosis, and those are unfortunately a more difficult situation because the government and corporations are generally the ones driving them, and they are culturally normalized.
Yes. I was offering examples. Again, having a difference of opinion is not a delusion.
If he meant mass psychosis, he should have said mass psychosis. And again, since he is not a public health scientist or any flavor of psych professional, he probably shouldn’t make those proclamations. And should probably call for a wellness check instead of posting on social media if he were truly concerned for their health.
I don't think this is all psychosis but more like extreme groupthink.
For people who are considered neurotypical, social coherence often overwrites reality. Its a mechanism for achieving consensus withing groups while spending the least amount of brain compute energy. Same goes for social metainfo tagged messages, they are more likely to influence reality perception, subconsciously. E.G: If a rich guy says you should be hyped the people who wanna get rich will feel hyped and emotional contagion can spread between people who belong to the same "tribe"
It's very visible for us atypical folk who can't participate well in groupthink at all
I guess at a company of seven, if two people are making the executive decisions and the two people are drinking the same AI kool-aid and the other five people are dutifully following these executive decisions, the whole company can be considered to be under this condition.
Having a difference of opinion can absolutely be a delusion. For example, I think you're probably not God. If you thought you were God, then we'd disagree, and you'd also be delusional.
I use that example because I have literally seen people fall into delusions of thinking they're God after talking to AI enough. That's shit is scary, for real.
was looking for this comment. this post is highly inappropriate and very inaccurate. this should be at the top. too many people are throwing around the word psychosis without knowing what it means. if someone is truely going through psychosis you get them help!
He uses "AI psychosis" as a description of people that are overzealous on AI. He is obviously not a person that can or would diagnose mental illness.
To the wider audience on HN the phrasing is pretty clear. An outsider with a tiny bit or intellectual charity wouldn't come to conclusions like you do.
Yeah, but AI psychosis can also be used to mean the stronger thing that the parent comment refers to -- something like AI-induced psychosis, which was how I originally understood the term:
Well, I agree with you that the parent comment is wrong inasmuch as it suggests we can't tell from context that mitchellh is using the term to mean "a value judgment" instead of "a form of psychosis". We can tell.
But I agree with the parent comment in that we shouldn't use the term "AI psychosis" to mean "a value judgment" instead of "a form of psychosis", because "AI psychosis" has already been used for 2.5 years to mean "a form of psychosis".
People would understand what he meant if he called someone awkward “autistic” too. It’s wrong to use medical terms as slang because it erases the actual meaning and disregards the lived experience of people who have been through the condition. People who have been around psychosis would come to the same conclusion. The majority of the population not having that exposure doesn’t make it right. It’s tasteless and inappropriate.
Using terms from domain metaphorically in another is a common and, I think, useful way of communication. While a view like yours has genuine merit, especially for a subset of the population who have experience personal or otherwise, with the medical condition, I think it's overly restrictive and counter productive to label it as outright tasteless and inappropriate.
I’ve had to do a ton of SQL stuff lately, which I haven’t really worked with since the late 90s. ChatGPT has been a godsend, not just for me, but for our only coworker who knows SQL well, whom I’d probably be bugging several times a day at my wits’ end.
But no one cares about those kinds of productivity gains. Just the ones that will completely replace us.
I'm the old school type who writes out a document that explains what I plan on doing in markdown even if it's generic like "a window with x and y buttons" and the logic flow and then use that to have ai write a plan with me before I send it off to execute it. This has worked super well.
I do enjoy giving the frontier models wacky projects that I can't even find examples of how to do online but I don't expect any results or need them and some have done really well with it while others fall on their face (models)
This is fine for a moderately sized query. When your queries start taking in 8 joins and 20 fields per table because you're running queries on Presto with 5 TB of data, not only is it drastically better at writing (because it doesn't mess up the fields), you can ask it to try the query 5 different ways to help you land on the most optimal.
This is a great example of AI tech-debt and fragility.
An eight-join query is going to be nigh on unmaintainable should the requirements change, leading to a change-break-change-break spiral as your preferred coding agent tries to fix its previous fixes.
Maybe the wise way to use AI would be to sort out the schema.
This feels wrong. 8 joins is almost certainly reporting stuff, not transactional. Contrary to what some SQL-averse devs think, 300 lines of SQL is actually more maintainable than the equivalent ~1000 lines of application code. It's also much faster. And I do think that's the real conversion, because SQL is a much higher level language than currently available application languages. It's also declarative in nature, which helps maintainance.
A highly normalized DB can easily end up with 8 joins required for some function. That's really not out of the question. "Sorting out" the schema then would be... denormalization, which is a thing, but you need to know why you're doing it. And I think 8 joins isn't enough of a reason.
I'm amazed you think that instead of using an LLM that someone will go buy a book and spend a week learning something that, judging by the fact that they last used it 30 years ago, likely won't be relevant for them soon.
It's not only that I rarely use it, it's also that it's ugly. It's Relational Cobol. It's as loveable as Oracle. The vendor specific dialects don't even agree on how to do recursive queries do they?
Unfortunately I am very good at forgetting things I resented having to learn, and SQL is definitively one of them.
If the AI's query pulled what I intended to pull, why should I care to understand the SQL any more than I should understand the Query Plan or the Machine Code?
SQL is (was?) one of my strongest skills, I enjoy it a lot, and I still reach for the LLM. It's just faster than me, and when it goes wrong (rarely) I can correct it in plain English.
I find SQL and data(bases) in general to be LLM’s Achilles’ heel. Databases are rarely under version control, so the training data only has one half of the knowledge.
My comments are more in the context of OLAP queries and other non-normalised data often queried via SQL.
I train non-LLM transformer models on (older and rarer) datasets, and automating the ingestion of sprawling datasets with hundreds of columns, often in a variety of local languages and different naming conventions adopted over decades, with quite a few duplicated columns…. The LLMs perform badly, it’s nigh impossible to test (for me as a user in prod) and it’s nearly impossible for the LLM companies to test (in training) to RLVR and RLHF this.
That's interesting - SQL is one of the places I find them the strongest - I think there must be an insane amount of training data out there for SQL. But mostly I'm asking them for ad hoc report queries. Nobody cares if they're bad SQL, they just want to know how many signups there were in March that didn't tick the marketing box. Sounds like you're pushing their capabilities a lot further than I am though - I just want to perform arbitarily complex queries on 3NF data.
Yeah not sure what this guy is talking about, LLMs excel with queries because the SQL language is pretty small in scope and its easy to test the output. Table structure and relationships are easy to feed to the AI.
I train non-LLM transformer models on (older and rarer) datasets, and automating the ingestion of sprawling datasets with hundreds of columns, often in a variety of local languages and different naming conventions adopted over decades
Just use an LLM to make a good knowledge base for the databases. Based on schema info and production queries. An agent can use that to write queries that work.
outsourcing their decision making and thinking to AI and not really about using AI itself
I use AI a ton and I'm having more fun every day than I ever did before
With respect, this is what makes me worry.
If someone is a user of AI, can they really tell the difference between "outsourcing" and "using"? I worry that a lot of people will start out well-intentioned and end up completely outsourced before they realise it.
there's a difference between having the LLM write stuff for you, checking it yourself, modifying it and merging it yourself, and just blindly trusting it to do whatever it wants.
You can ask an overseas consultant to prepare a prototype of your program for you, check it yourself, and only use it if it passes your standards, or fire your whole dev team and blindly trust the overseas bodyshop.
The difference, at least from my point of view, between "using" and "outsourcing" is that in the former case, you're still responsible for the output, you view it as a tool that helps in some use cases, vs just giving up all control.
It’s really frustrating too because even just the plain language translation and pattern matching aspects have such incredible uses.
As a cybersecurity IR professional being able to have a constantly logging counterpart who’s also able to go run queries and check logs on its own is an incredible speed boost.
I can just throw it a finding and have it slot it into a timeline and make notes.
I can toss it something mildly interesting to chase down while I focus on the obvious activity.
So many things that don’t involve having it “think” for you and keep you in the front seat.
But all of that is constantly overshadowed by these companies pushing the automation or “reasoning” aspects more and more and the sycophants who screech that it’s perfect and can do no wrong when every serious users experience is that “yes, it definitely can, often to catastrophic effect”.
The worst part of AI is that the time to produce software has become entirely unpredictable. "If Claude is randomly good at this, and happens to be up today, it will take me about 3 hours. If Claude is randomly bad at this task, or has downtime, 2 weeks"
I didn’t think just offloading your thinking to AI was AI psychosis.
To me AI psychosis is the handful of friends I’ve had who have done things like have a full on mourning session when a model updates because they lost a friend/lover, the one guy who won’t speak to his family directly but has them talk to ChatGPT first and then has ChatGPT generate his response, or the two who are confident that they have discovered that physics and mathematics are incorrect and have discovered the truth of reality through their conversations with the models.
But language is a shared technology so maybe the term is being used for less egregious behavior than I was using it for.
friends I’ve had who have done things like have a full on mourning session when a model updates because they lost a friend/lover
I mean, isn't that the natural and expected response? An AI company sold them a relationship with a chatbot and at least some their social/romantic needs were being met by that product. When what they were paying for was taken from them and changed without warning into something that no longer filled that void in their life why wouldn't
they morn that loss?
The fact that they were hurt by that sudden loss is totally healthy. It's just part of moving on. The real problem was getting into an unhealthy relationship with a fictitious partner under the control of an abusive company willing to exploit their loneliness in exchange for money.
Hopefully they now know better, but people (especially desperate ones) make poor choices all the time to get what's missing in their lives or to distract themselves from it.
I mean, isn't that the natural and expected response? An AI company sold them a relationship with a chatbot and at least some their social/romantic needs were being met by that product. When what they were paying for was taken from them and changed without warning into something that no longer filled that void in their life why wouldn't they morn the loss of that?
Ah, I forgot about the ai relationship companies. No this guy was using the browser based ChatGPT for coding and ended up in love with the model. No relationship was sold at all.
Wow, okay. Reading a whole relationship into that sort of interaction is way less reasonable, although now that I think about it a somewhat similar thing happened to Geordi La Forge once...
It’s not just way less reasonable, it’s depressing. I feel like a new drug was released and I’m watching multiple friends succumb to it.
Seeing people whose thoughts and opinions you used to respect turn into objectively insane people has been some of the worst times I’ve had since graduating during the Great Recession in terms of how stressful it’s been.
I'm curious how to best define what AI psychosis actually is.
My understanding is that regular psychosis involves someone taking bits and pieces of facts or real world events and chaining them into a logical order or interpolating meanings or explanations which feel real and obvious to the patient but are not sufficiently backed by evidence and thus not in line with our widely accepted understanding of reality.
AI psychosis is then this same phenomenon occurring at a more widespread scale due to the next-word-prediction nature of LLMs facilitating this by lowering the activation energy for this to happen. LLMs are excellent at taking any idea, question, theory and spinning a linear and plausibly coherent line of conversation from it.
I’m not even claiming negatives against you now. You were a green account in my UI and it said it was created 1 hour ago when I made my initial comment.
The UI is now showing different information and a comment from years ago so I am genuinely curious if it’s a bug in the forum software.
If you’re trying to say that the wealthy have it just as hard as the poor I would ask you to step off a cliff under the claim that gravity doesn’t exist.
Would they, though? Current AI stuff is delivering something functionally nonexistent in human history before this: absolute sycophancy, 24/7, on demand, for anyone who wants it. People joke about the wealthy becoming detached from reality because of yes-men, but this is a stage beyond even the capability of the most dedicated brown-noser.
I agree that these people had mental health issues. I think if they got to billionaire level and were surrounded by yes men they would have the same reaction.
The difference nowadays is you can get the same surrounded by yes men experience for only 20 dollars a month so a lot more of the people who are primed for this sort of breakdown are now being exposed to it due to the decrease in cost.
That is a possibility indeed, I agree. There are mild mental issues that might have gone under the radar before and are now magnified by the AI sycophancy. I'm no mental health expert so I can't really tell but it does make some intuitive sense.
Edit: but at the same time there are issues that were always there and Just manifestate in new ways. A bit like addiction, you can have an addictive personality already, but if you get addicted on heroin is much much worse that on tobacco.
The way I put this to myself is that AI gives “correct correct answers and incorrect correct answers”.
They almost always generate logically correct text, but sometimes that text has a set of incorrect implicit assumptions and decisions that may not be valid for the use case.
Generating a correct correct solution requires proper definition of the problem, which is arguably more challenging than creating the solution.
It’s simpler than that - it’s a guessing machine that has superior access to a whole load of information and capacity to process at a speed at which we humans cannot compete.
Does it make it better than us? No because ultimately the thing itself doesn’t ‘know’ right from wrong.
Yeah, very often the issue is that some context is missing. It'll say something true, but which misses the bigger point, or leads to a suboptimal result. Or it interprets an ambiguous thing in one specific way, when the other meaning makes more sense. You have to keep your wits about you to catch these things.
It's an incredible tool but it's also very derpy sometimes, full of biases, blind spots etc.
which is arguably more challenging than creating the solution.
This hasn't been the case in my experience. Devising a correct solution without a definition of the problem is impossible because you wouldn't recognize a correct solution without a definition. Often you discover the problem definition by exploratory programming and trial and error on solutions, but LLMs are still good for process this too. Arguably better because they type faster so you can iterate faster!
Though there is some overlap in software development. Like for example using heavy-weight dependencies, that try to follow the one size fits all approach, when one could use a much simpler, faster or even no dependency at all. The LLMs will readily suggest quickly adding that huge dependency, that is mentioned in beginner tutorials. Or suggest to use regex for parsing HTML.
(Real example, had this from Kimi 2.6 recently, lol.)
this author suggest its essentially the same risk https://www.poppastring.com/blog/what-we-lost-the-last-time-.... i feel its heightened because execs and leaders are absolutely salivating over the opportunity to fire thousands of humans with no regard for the cognitive debt that comes from outsourcing thinking to ai.
While you have to think about things objectively no matter what, when I start researching topics like physics, using AI as suggested in that article has proven very useful.
if you just prompt the AI and believe what it tell you then you have AI psychosis
This is the right definition. LLM outputs have undefined truth value. They’re mechanized Frankfurtian Bullshiters. Which can be valuable! If you have the tools or taste to filter the things that happen to be true from the rest of the dross.
However! We need a nicer word for it. Suggesting someone has “AI psychosis” feels a bit too impolitic.
Maybe we reclaim “toked out” from our misspent youths?
e.g. “This piece feels a little toked out. Let’s verify a few of Claude’s claims”
I wouldn’t say they have an undefined truth value. Their source of truth is their training data. The problem is that human text is not tightly coupled to the capital T truth.
Several people I know have already gone through phases like this. When you're doing it alone there is a moderating factor when their friends and family start calling them out on their behavior or weird things they say.
I can't imagine how bad it would be if your employer started doing this from the leadership. You'd be pressured to get on board or fear getting fired. Nobody would be trying to moderate your thinking except your coworkers who disagree with it, but those people are going to leave or be fired. If you want to keep your job, you have to play along.
I suspect we're going to see this in many corporate environments soon, if we aren't already
your coworkers who disagree with it, but those people are going to leave or be fired.
Personally I expect that I will be this person soon, probably fired. I'm not sure what I will do for a career after, but I sure do hate AI companies now for doing this to my career
this is exactly what is happening. instead of building true AI culture around thoughtful adoption of AI strengths while defending against weaknesses, they're coming up with bullshit heuristics like "every repo has a CLAUDE.md", watching private token usage dashboards, and terrorizing everyone into doing it (or lose your job).
this leads to naive AI adoption, which is the worst of both worlds (no real speedup, out sourcing thinking, ai slop PRs, skill rot).
I have a friend that is a junior in a security-oriented sys-admin/network engineer type role. They have been doing the job for only a bit over a year. No background in programming.
Their entire organization has been handed Codex/Claude and told to "go all in on AI" and "automate everything". So the mandate is for people that do not know how to code and have the keys to the castle to unleash these things upon their systems.
This is at a large organization with tens of thousands of employees.
I am waiting with bated breath for the ultimate outcome!
From what I have seen, most corporate it security people are at a service desk level at best. They are tool runners who don't really understand what the tools spit out, they just go bug other teams about it.
I agree with you, except it isn't even good at writing code. Almost every time that you get an LLM to write a bunch of code for you, it has mistakes in it. The logic isn't right, the API calls aren't right, the syntax isn't right (!). That problem hasn't yet been fixed and it looks as though it never will be. That means that every line of code it generates, you have to review, because even if 95% of the code is correct, you need to find the 5% which isn't. But if you have to do that, it becomes slower than just writing the code yourself. As people have pointed out over and over again: typing in the code was never the part that took time. So I don't agree that LLMs are really useful for writing code.
LLMs are good at producing code that seems plausible at first glance and appears to work, but it never really does. And when trying to fix things, you discover 7 slightly different ad hoc implementations of the same thing, with their own weird edge cases and behaviors. And you likely miss 4 more. There is no intention or coherence behind any of it.
if you just prompt the AI and believe what it tell you then you have AI psychosis. You see this a lot with financial people and VC on twitter
I'm seeing it with lawyers, too. Like, about law. (Just not in their subject matter.) To the point that I had a lawyer using Perplexity to disagree with actual legal advice I got from a subject-matter expert.
when you outsource thinking to AI, you get that magical speed up. the agent is making decisions for you, so things move at agent speed. it often makes decisions without telling you, and the final "here's the plan" output often requires you to understand the problem at great depth, which requires return to human speed, so you skim and just approve.
the trick is to be mindful, aware, and deliberate about what decisions are being outsourced. this requires slowing down, losing that absurd 10x vibe coding gain. in exchange, youre more "in-the-loop" and accumulate less cognitive debt.
find ways to let the agent make the boring decisions, like how to loop over some array, or how to adapt the output of one call into the input of another.
make the real decisions ahead of time. encode them into specs. define boundaries, apis, key data structures. identify systems and responsibilities. explicitly enumerate error handling. set hard constraints around security and PII.
tell the agent to halt on ambiguity.
a good engineer will get a 2x or 3x speedup without the downsides.
find ways to let the agent make the boring decisions, like how to loop over some array, or how to adapt the output of one call into the input of another.
Those kind of advice ultimately don't matter. If you're familiar with a programming project, you'll also be familiar with the constructs and API so looping over an array or mapping some data is obvious. Just like you needn't read to a dictionary to write "Thank you", you just write it.
And if you're not, ultimately you need to verify the doc for the contract of some function or the lifecycle of some object to have any guaranty that the software will do what you want to do. And after a few day of doing that, you'll then be familiar with the constructs.
make the real decisions ahead of time. encode them into specs. define boundaries, apis, key data structures. identify systems and responsibilities. explicitly enumerate error handling. set hard constraints around security and PII.
The only way to do that is if you have implemented the algorithm before and now are redoing for some reason (instead of using the previous project). If you compare nice specs like the ietf RFCs and the USB standards and their implementation in OS like FreeBSD, you will see that implementation has often no resemblance to how it's described. The spec is important, but getting a consistent implementation based on it is hard work too.
That consistency is hard to get right without getting involved in the details. Because it's ultimately about fine grained control.
If there's one thing I know about users is that they're never certain about whatever they've produced.
I am starting to come around to a similar sentiment. I have seen several large projects cook now for almost a year are not done. These are not trivial projects but the leads are heavily using ai at every opportunity.
I wasnt before but I am 100% confident that AI has done nothing to speed the delivery. It hasnt slowed it down either. It is a wash. The job is more miserable though.
What I'm seeing is a little eternal September of support tickets about programs that fail to interface the JSON API of a customer of mine. The API is always allucinated. In the best case there are out of place attributes. Often they don't exist at all. I've seen x, y, width, height when we have only top and left. Of course no human read the documentation. Those are probably founders vibe coding a client without the technical competence of understanding the API doc on Postman. That is understandable. Unfortunately they don't even have the competence of pointing their AI to Postman in the right way. My custumer assessed that they will always find a way to do a mistake despite any mitigation from our side. What I do is replying to those tickets with line by line comments of the allucinated JSON. I never talk about AIs because I might hurt the pride of some of them and, who knows, some little mistakes could be from real junior developers. Sometimes the tickets are followed up by more puzzled ones, sometimes they fix the problem. Probably they copy and paste my reply to their bots.
I've heard the same thing mentioned by a close friend building integrations. They are helping/supporting real use cases but they decided not to help vibe coder founders without an understanding of how APIs work etc. It's just too big of a gap to cover even for larger companies with strong support.
Seeing this too. Customer support tickets are all AI now. The random bolded words, the em dashes, they way where if you KNOW what is actually happening, they are slightly off or just WAY off.
but if you just prompt the AI and believe what it tell you then you have AI psychosis.
No it isn't. Do you believe what teachers told you in school? Yes? Well, I guess you're suffering from just normal psychosis!
I don't understand how people don't understand that people offer unreliable information too. We learned about the tongue map in school as kids - many kids still learn that in school today. It's still BS regardless whether it was told to you by a teacher or AI.
You don't suffer from psychosis for believing a source of information, you're simply mistaken. You need a more critical eye to assess what you're told in general, not just AI.
Do you believe what teachers told you in school? Yes?
Nope. At least, not without proof. That would, IMO, be kinda crazy. We could argue semantics - maybe “stupid” would be a better word? Lacking in critical thinking skills? Whatever “it” is, it isn’t good.
There's a huge difference between a teacher giving outdated information representing what was once our (or at least their) best understanding of the world, and a chatbot that just randomly makes up things for no reason while insisting that it's all true.
Also, a good teacher should be encouraging the development of critical thinking skills and correcting your errors, while AI will just tell you how brilliant you are when you wrongly tell it about how you've just invented a new form of math or disproved a scientific theory you barely understand in the first place.
Not all BS is the same, just as not all sources are equally unreliable.
I would say writing it myself is more enjoyable (in some cases). But I quite understand that I am not paid to enjoy myself. I'd say it's quicker getting AI to do it and reviewing. I believe the outcome is no worse on average. So yes, that's my chosen approach.
I've been strictly using LLM's to either push stuff that I've done plenty times before and are mostly boilerplate or have zero value for writing them by hand (not even educational), and I always ENSURE that they work on stuff that are easily verifiable and proven incorrect with my existing knowledge or a few minutes of googling.
LLMs can do advanced math and coding, which involves logic, so they are definitely capable of using logic. Which is what most people call reasoning.
So "LLMs are incapable of reasoning, they are just pattern matchers" is wrong. A lot of logic _is_ pattern matching, BTW. Like, syllogisms - deductive reasoning - do you think LLMs are incapable of that?
The thing you're referring to is that LLMs are trained to produce an answer which a human would like, i.e. they aim to produce plausible rather than correct answers.
So it's not so much a mental deficit as a different goal. Trusting LLM blindly is definitely dangerous, but dismissing it as useless for anything by code is rather wrong.
Pattern matching is hardly what distinguishes human from LLM - if you ask somebody a question about policy, for examples, chances are they'd just recite something they heard somewhere, never really thinking about it from first principles.
I’ve been talking to a lot of engineers about how they use AI in their day to day and it’s dramatically different than what you see from the hypers.
The vast majority use one agent at a time and careful step through code. The main benefit they report is often about researching the codebase and possible solutions.
Garry Tan has been the primary crusader for AI driven decision making. I'm sure his position is more nuanced, but his twitter driven communication makes him appear like a caricature of a man in AI psychosis.
When the head of YC champions AI driven decision making, companies will inevitably be influenced into doing exactly that. It's unfortunate, because AI is generational technology and the hyperbole distracts from the real sea change occuring in labor markets everywhere.
I don't think using AI to write code is AI psychosis or bad at all, but if you just prompt the AI and believe what it tell you then you have AI psychosis.
Today's frontier models are genuinely useful as rubber ducks or grunt units. They are horrible for actual problem solving. These tools are not capable of actual reasoning. They will happily crap out a broken, untyped, untested Next.js monstrosity with no discernible architecture. They will build esoteric shell scripts to perform operations that could be done idiomatically and simply with tools already in your codebase. They will tell you to walk to the car wash then have the car wash valet your car back to you when confronted with the flaw in their logic. They will validate incorrect beliefs like ketchup being an acceptable hot dog condiment or the notion that "The Red Hot Chili Peppers" make good music. They have no taste, no anima, no drive.
Rule #1: Do not anthropomorphize the LLM. It is a million monkeys at a million typewriters piped into a digital sieve. I don't know how or why people place such trust in them while bemoaning other technology in our lives for being so broken ("my algorithm [sic] only shows me X", "the new iPhone update sucks", etc). If everybody followed this rule then the deluge of emoji-ridden hokum pouring into Slack workspaces and GitHub PRs around the world would cease but I'm not holding my breath.
Part of the psychosis are AI usage mandates, where companies require a certain amount of LLM usage per worker. Of course these things are useful, but forcing them on workers is psychotic.
Mitchellh is on to something. Some of the AI products I've seen seem like psychosis hallucinatory fever dreams, using terms and concepts that have no meaning. Funding? $50,000,000 pre-seed.
This is a critical communications issue that is becoming what I believe the defining characteristic of "This Age": nobody knows how to discuss disagreement, and because it cannot even be discussed communication ends, followed by blind obedience, forced bullying, retreat and abandonment. This is going to be a hell of a ride, because nobody can really discuss the situation with a rational tone.
We're definitely in the mess around phase of AI adoption.
I don't think it's super clear what we'll find out.
We've all built the moat of our careers out of our expertise.
It is also very possible that expertise will be rendered significantly less valuable as the models improve.
Nobody ever cared what the code looked like. They only ever cared if it solved their problem and it was bug free. Maybe everything falls apart, or maybe AI agents ship code that's good enough.
Given the state of the industry were clearly going to find out one way or the other, hah!
I don't think it's super clear what we'll find out
I think some companies will find out that their senior engineers were providing more value and software stability than they gave them credit for!
Corporate feedback loops are very slow though, partly because management don't like to admit mistakes, and partly because of false success reporting up the chain. I'd not be surprised if it takes 5 years or more before there is any recognition of harm being done by AI, and quiet reversion to practices that worked better.
The AI psychosis is not the anti-opinion to the use of AI.
I use AI coding tools every day, but AI tools have no concept of the future.
The selfish thinking that an engineer has when they think "If this breaks in prod, I won't be able to fix it. And they'll page me at 3AM" we've relied on to build stable systems.
The general laziness of looking for a perfect library on CPAN so that I don't have to do this work (often taking longer to not find a library than writing it by hand).
Have written thousands of lines of code with AI tool which ended up in prod and mostly it feels natural, because since 2017 I've been telling people to write code instead of typing it all on my own & setting up pitfalls to catch bad code in testing.
But one thing it doesn't do is "write less code"[1].
I use AI coding tools every day, but AI tools have no concept of the future.
The selfish thinking that an engineer has when they think "If this breaks in prod, I won't be able to fix it. And they'll page me at 3AM" we've relied on to build stable systems.
Maybe it's just my prompt or something but my coding agent (Opus 4.7 based) says things like "this is the kind of thing that will blow up at 2am six months from now" all the time.
It's really inconsistent though.. it takes shortcuts and leaves todos all the time without really calling it out explicitly, you have to pay close attention.
I don't doubt there are companies totally misusing coding agents and LLMs in production. There are also real companies with real revenue and solid architecture using LLMs to deliver products. There are also companies with real revenue and rapidly accumulating tech debt.
Eventually the companies that can't cope with undisciplined engineering will succumb to unacceptable reliability and be outcompeted, just like in the "move fast and break things" era.
"Just use autoresearch and it will fix your app's memory leaks in an hour" is what I was nonchalantly told by someone who has never written a line of code ever.
I guess what I relate to the most is how dismissive people get about real software engineering work.
I may have skill issues, but I am yet to reach the level of autonomous engineering people tend to expect out of AI these days.
I'm going through a mixed experience regarding this, personally.
Management is really pushing AI. It's obnoxious, and their idea on how it fits into my team's job specifically is completely, hilariously detached from reality. On the off chance someone says something reasonable, unless it fits the mold, it's immediately discarded. The mold being "spec driven development". We're not even a product team for crying out loud. I straight up started skipping these meetings for the sake of my sanity. It's mindwash, and it's genuinely dizzying. The other reason I stopped attending is because it ironically makes me more disinterested in AI, which I consider to be against my personal interests on the long run overall.
On the flipside, I love using Claude (in moderation). It keeps pulling off several very nice things, some of which Mitchell touched on in this post (the last one):
- I write scripts and automation from time to time; Claude fleshes them out way better with way more safety features, feature flags, and logging than I'd otherwise have capacity to spend time on
- Claude catches missed refactors and preexisting defects, and does a generally solid pass checking for defects as a whole
- Claude routinely helps with doing things I'd basically never be able to justify spending time on. Yesterday, I one-shotted an entire utility application with a GUI to boot, and it worked first try; I was beyond impressed.
- Claude helped me and a colleague do some partisan cross-team investigation in secret. We're migrating <thing> and we were evaluating <differences>. There was a lot of them. Management was in a limbo, unsure what to do, flip-flopping between bad options. In a desperate moment, I figured, hey, we kinda have a thing now for investigating an inhuman amount of stuff in detail - so I've put together a care package for my colleague with all our code, a bunch of context, a capture of all the input data for the past one week, and all the logs generated. Colleague put his team's side of the story next to it, and with the help of Claude, did some extremely nice cross-functional investigation. Over the course of a few weeks, he was able to confirm like a dozen showstopper bugs, many of which would have been absolutely fiendish if not impossible to fix (or even catch) if we went live without knowing about them. One even culminated in a whole-ass solution re-architecturing. We essentially tore down a silo wall with Claude's help in doing this.
So ultimately, it really is a mixed bag, with some really deep lowpoints and some really nice higlights. I also just generally find it weird that a technical tool [category] is being pushed down people's throats with a technical reasoning, but by management. One would think this goes bottom up, or is at least a lot more exploratory. The frenzy is real.
This will be pushed down from people, who will have no deep understanding of it. But it does check some boxes in an ISO certification.
Well, now you must to work with a confusing tool which slows you down. You are not allowed to use claude directly anymore, because someone heard that mythos is really bad for security. But hey, the tool integrates well with Jira!
You hate every second working with this thing. All the joy you had with explorative coding is forever gone, which was the sole reason you entered this field.
Deep inside you know that you can't change your job, because every other employer will cut its workforce as AI removes all manual labor of a software engineer and reduces risk to a minimum.
Oh, now we can finally move all those jobs to india without risk and shareholders will love it! How awesome is that! Wait, do we still need that guy in cubicle 42, who bitches and moans about AI every day? Nah...
Totally agree with one shotting GUI tools. I especially have liked it to create a single-file web app, and then open it with Chromium locally (no web server needed).
In my case, it built a tool for splitting sounds and a tool for defining hitboxes for a game. Tools made exactly for more workflow. Wild times.
Sounds pretty accurate. Bunch of comments on this thread sound like AI is some kind of a new doomsday cult. The most annoying thing I find personally is that all engineering principles are getting crushed by non techies. Management counting token usage, forcing agent use, reducing headcount in the name of productivity gain. Devs building bridges but nobody knows what the bridge is, what are the standards to which it was built, how it works and how to maintain it. VCs counting extra money claiming chasing the holy profit is the future. The abundance of engineering apathy is disturbing.
I think AI rescue consulting is going to be come a significant mode of high value consulting, similar to specialists who come in to try and deal with a security breach or do data recovery.
Purely AI written systems will scale to a point of complexity that no human can ever understand and the defect close rate will taper down and the token burn per defect rate scale up and eventually AI changes will cause on average more defects than they close and the whole system will be unstable. It will become a special kind of process to clean room out such a mess and rebuild it fresh (probably still with AI) after distilling out core design principles to avoid catastrophic breakdown.
Somewhere in the future, the new software engineering will be primarily about principles to avoid this in the first, place but it will take us 20 years to learn them, just like original software eng took a lot longer than expected to reach a stable set of design principles (and people still argue about them!).
It's kind of like producing code is becoming more like farming.
We didn't create the dna we rely on to produce food and lumber, we just set up the conditions and hope the process produces something we want instead of deleting all the bannannas.
Farming is a fine an honorable and valuable function for society, but I have no interest in being a farmer. I build things, I don't plant seeds and pray to the gods and hope they grow into something I want.
Prayers are for weather. Pretty much all farmed plant, animal, and fungus species have been selectively bred or genetically modified. Farmers know what's going to grow.
Farming has merely a lot of study and input into the process, very little actual control and no determinism at all. We know how to improve chances is all. The fact that we breed and "engineer" is like a drop in the bucket.
You might grow corn, or you might grow defective unusable corn and/or any number of other things like locusts or fungi or other plants that decide to grow in the place where you planted corn. Sure, the corn seeds will not produce ball bearings. Genius observation. There are about an infinity of other things that can and do happen besides that.
Planting is merely setting up the conditions. We didn't write the dna, we couldn't write the dna if we wanted to because we are an infinity away from understanding all the actual processes that descend from the dna. And when we utilize the dna that we simply found and didn't and couln't hope to write, it's always, at best, a case of hoping it goes right again this time.
Tell me you've never done any farming without telling me you've never done any farming. There is certainly risk in the business due to market fluctuations, weather, natural disasters, disease, and pests. But the final product is highly deterministic. Almost all genetic variability has been expunged from major food production species in a relentless pursuit of predictable yield. Everything looks and tastes the same. We can debate whether that's a good thing but it is the reality for most farmers.
If it was deterministic, there would be no such thing as blights and other forms of failures. There would be no problem with the bannannas, or coffee or wine grapes. There would be no such thing as a critical few days of the entire year where if anything goes wrong you lose the entire year because it was too humid or too cold or your equipment was out of commission for a week. The bees wouldn't matter at all.
Even when it works, even if you put in a lot of work and experience and understanding, it still just worked by itself and it's just good luck every time.
Purely AI written systems will scale to a point of complexity that no human can ever understand and the defect close rate will taper down and the token burn per defect rate scale up and eventually AI changes will cause on average more defects than they close and the whole system will be unstable.
Wow, it’s true, AI really is set to match human performance on large, complex software systems! ;)
The origin of 'dark DNA' begins to make more sense through this sort of lens, except the system somehow maintained a level of compensation to fix all its flaws.
Humans who have been writing systems like that for many years know how to maintain and modify them successfully. It’s just that our industry has a bias towards youth who don’t think they have anything to learn from those who came before them.
How do you explain to a junior this pile of messy code isn’t crap but is actually years of integrated knowledge ? That the most common principles discussed in computer science (OOP, SOLID, DRY etc.) are actually just little guides that aren’t to be taken to the extremes ?
A decade ago, I was sitting in on a meeting about a rewrite and, before I could say anything, someone in the first year of her career asked why anyone thought a rewrite would be any cleaner once all the edge cases were handled. Afterwards, I asked her where she learned this. She said "I don't know, it just seems kind of obvious." She went on to be a great engineer and is now a great manager.
I work on internal facing software and every rewrite I've seen in 20 years suffers from the same symptoms. The code/system is a mess because it has been exposed to reality for a decade. Reality is messy. That's why they pay us money, believe it or not.
Greenfield guy comes in, promises the world, and starts from some first principles white papered architecture. It's really lovely until they onboard the first user. Then they slowly commit all the "sins" (features that drive revenue) of the first system.
The firm is stuck supporting N systems indefinitely because the perfect new system takes so long to cover even 30% of the original system use cases, that management takes a flier on.. bear with me.. a second rewrite. Now they have 3 systems.
I've seen more 3rd systems than I've seen actual decommissioning of original systems into a single clean new system.
The answer is chipping away, modularizing, and replacing piecemeal Ship of Theseus style. But that does not drive big hires and big promotions.
My team lead has worked on the same software for 30 years. He has the ability to hear me discuss a bug I noticed, and then pinpoint not only the likely culprit, but the exact function that's causing it.
I do the same thing in a project I’ve worked on for 25 years. I’ve had mediocre at best results with AI. It’s useful to discuss concepts with, but the code never handles the nuances of the edge cases.
Yep this is like comparing master craftsmanship with a production line. You're gonna get good attention to detail and a masterpiece from one, and a limited thing that will break after few years from the other. But for majority of use cases the second one is enough. And pointing out the master craftsmanship is "better" is besides the point.
And with one you need to train a guy for 25 years and with the other you need plan mode for a few minutes and then it runs 24/7.
Yeah... in my experience people who code like that 'successfully' make modifications that fix an immediate problem while kicking another bug or two further down the road in a never-ending sunk-cost-fallacy of job security...
I have really tried as an "old" person in the field to try and pass on the stuff I've learned, but "craft" and such really has absolutely no home in modern dev culture. The people who care about history, the craft, etc. are increasingly rare.
Shelve it with the Jurassic Park version where John Hammond builds a safe, profitable theme park, and The Andromeda Strain that gives people the sniffles.
Frankly this is what everyone is counting on whether they know it or not. The question though is not “will the models get good enough?”. The question is does the repo even contain enough accurate information content to determine what the system is even supposed to be doing.
Every frontier model from each major US lab is cheaper than their frontier model this time a year ago with the exception of Anthropic whose pricing has remained exactly the same.
People are often skeptical when I say this, but there's simply no guarantee that it's possible in principle to clean up a bad architecture. If your system is "overfitted" to 10,000 requirements from 1,000 customers, it may be impossible to satisfy requirements 10,001 through 10,100 without starting over from scratch.
It's really not that big of a word. The CAP theorem shows that as few as three reasonable-sounding requirements with no obvious conflicts can be impossible to satisfy simultaneously. (User needs will start more flexible than strict mathematical requirements, of course, but once people start to build production workloads on top of your systems that flexibility is radically reduced.)
I really am surprised that people on a heavy CS themed forum still have trouble grasping this.
Imagine the year is 1995, C exists, but some guy out there is working on essentially what modern Python is. He says to you "check out this language, you can just import stuff, and use it and dynamically modify anything at run time". You can probably come up with hundreds of arguments about things that could go wrong, like memory clean up, threading, e.t.c, but turns out, incrementally, they were all solved and we have the modern Python that basically is good enough to build these large LLM models.
Now imagine modern programming and computing is what C was back in 1995, and AI use is that guy building the Python code.
The point is that in the future, AI will be able handle things like missing databases just like the modern high level dynamic languages can import a library to handle whatever you want.
I can't tell if you're being facetious, but a future AI really may be able to fill in a missing database. Like, if it knew some of the entries, it could infer the rest.
Wow - imagine being able to infill a geophysical database with the dullest possible milquetoast totally expected signal derived from the NASVD most common eigen vectors.
The infill will look seamless.
And entirely lack any actual strikes of interest - the outliers are exceptional signal and the entire raison d'etre for building such a database.
Jeez, if AI can just infill where the gold is, why even bother to look in the first place.
"clean up" dropped databases, compromised computers or leaked personal data?
For each of those things, you can right now build an agent that handles all of that. Or use a large frontier model with enough context to build code that ensures all of those edge cases are handled.
Future coding will essentially be like this. The concepts of dynamic vs compiled language will shift towards having frontier edge models put together code versus small runtime edge models dynamically processing data.
You can imagine anything you want, but it’s not an argument - you could apply this to anything. “Python was successful after a dubious beginning so NFTs will be successful”
Also, Python does not build or run large language models. It orchestrates C code that does that, and it was probably good enough to do that in 1998.
Highly dynamic languages existed for decades prior to 1995, Python was not particularly innovative in its features at the time. There were also countless languages more feature-rich than C being used for development at the time.
The biggest change that happened was that hardware kept getting better and it became feasible to use garbage-collected languages everywhere including really inefficient implementations like CPython.
That being said, 30 years later Python is still slow as shit even compared to other dynamic languages and runs into all kinds of scaling issues when used for anything serious. And everywhere that performance matters, software continues to be written in typed, compiled languages including C (but also C++, Rust, Go, etc.). Even in ML, Python chiefly acts as a thin wrapper and glue language for high performance CUDA libraries (aka C and C++).
So your historical analogy is mostly anachronistic.
No, you just don't have a grasp on reality. For example, you claim that Python runs into scaling issues for anything serious, but you are blissfully unaware that youtube and uber both run python backends. Nobody cares that its "slow" by whatever metric you consider. Its fast enough. The metric that matters is developer time not compute time, because the former is vastly more expensive. Python and Node are the number one languages on github for a reason. And you are vastly deluded on how many jobs there for C++ and Rust devs lol.
In the future, you won't be dealing with strings, json, or apis. You will be importing agents, and giving them brief instructions, either in plain English or in some intermediate language higher than Python that is more brief. Wanna deal with database reliability ? Import database agent and give it brief instructions on what you want to manage. Just like you mention, right now Python is the wrapper for low level libraries, because everyone who is doing work in ML doesn't want to waste time making sure their C Cuda kernels compile. In the same way, nobody is going to care if they get the API headers right, or if their strings are correctly parsed when you can just invoke a dedicated LLM (which will likely be highly specialized small model able to run on local hardware) to do all that.
You can scream and cry as much as you want how that is bad, how its slow, but nobody is going to care because shit is going to get built faster. Ever notice how despite the massive layoffs across tech, there isn't service degradation in any sector? Good luck trying to sell your Rust skills in the future lol.
Someone responded to a previous comment of mine [0] positing a Peter principle [1] of slopcoding — it will always be easier to tack on a new feature than to understand a whole system and clean it up. The equilibrium will remain at the point of near, but not total, codebase incomprehensibility.
Yes. And as the models get better, it works better. But at one point you do have to understand the code because it's also just guessing as to what your actual intentions are.
It doesn't know what mess you want to clean up. A lot of times AI just starts making up new patterns on top of other patterns and having backwards compatibility between the two. How does it know which one you actually like?
A non-technical friend of mine has just won some hospital contracts after vibecoding w/ Claude an inventory management solution for them. They gave him access to IT dept servers and he called me extremely lost on how to deploy (cant connect Claude to them) and also frustrated because the app has some sort of interesting data/state issues.
What concerns me about this is that as these stories multiply and circulate people will just completely stop buying software/SAAS from startups, because 90% or more will be this same thing. It will completely kill the market.
Or you end up with a certification process, which will of course introduce it's own problems but startups doing things the right way and not just "moveing fast and breaking things" can thrive.
Those are custom software or heavily customized implementations of ERP and similar systems for very large organizations. I’m talking more about the SMB market where today it’s possible for a small team to carve out a niche and make a nice living or even bootstrap a venture that competes with a large player that has poor UX or antiquated feature designs.
The reason Oracle can continue failing at those massive projects is simple: everyone fails at them routinely and often it’s the customers fault.
I used to gripe about various ERP companies but after having dealt with enough, yeah, that's just what the world of ERP systems is like. You will spend your time even with the best of them desiring to scream endlessly at everyone who works there. And they also know your pain but are powerless to help.
The reason Oracle can continue failing at those massive projects is simple: everyone fails at them routinely and often it’s the customers fault.
It's even simpler. Youre not paying oracle for some delapidated HR system. You're paying for the legion of accountability that is their on-site engineers to fix stuff for you when things screw up. You're essentially subscribing to a team of engineers you don't need to directly pay salary and benefits to.
People who think you can out efficiency that kind of accountability don't understand how large orgs think.
But the Torment Nexus is such an interesting technical challenge! and I don’t personally torment people: I just move protobufs around! - Software Engineer #1 and #2 excuses
On January 3, 2022, the jury found Holmes guilty on four of the seven counts related to defrauding investors: three counts of wire fraud, and one of conspiracy to commit wire fraud. She was found not guilty on four counts related to defrauding patients
This hospital will learn some hard lessons. I hope their backup strategy is good. I'm surprised they can field software from an entity that isn't SOC2 & HIPAA certified.
No worries! At worst, the contractor can just tell Claude to make sure the hospital knows they're appropriately certified. And the hospital can use Claude to make sure the certs are valid. Everybody wins, except the ones who end up dead. Or with their health destroyed.
I was going to say to open yourself up as a contractor and scape some of the money off top. But it sounds like you dint need that opportunity.
That sadly does seem to be the trajectory of 5-10 years from now, though. I can't speak to if "AI is the future" of 30+ years from now, but these coming years sounds rife for "janitors" to clean up all the slop being produced by newly empowered idea guys
I work at a university and we still have some workstations that need IE as well, for a healthcare vendor app that needs ActiveX. Up until recently we even had some machines running Windows 7.
This is going to happen all over. Company I'm currently contracting with has gone AI everything (aka technical debt hell), and they're gonna suffer for it. I'm glad my consulting contract ends in 2 months. I don't want to be around for the crash
As a cybersecurity IR professional as much as I hate to see this happen to a hospital this kind of thing is responsible for essentially tripling my income over the last 3 years.
As a SWE that has only ever worked for an employer or on his own projects, this makes me wonder: how would someone even get such a contract? Did this person already have a consulting business? Do you just call up random hospitals and ask if you can demo an inventory management system for them? Did this person already know people at the hospital? I know technical folks that do independent consulting, but even with a vibecoded product, how is it that anyone can just get such a contract?
People really have a misconception about the sums of money that companies operate on on a regular basis. If you are a people person and know essentially how to sell yourself, you can "scrape" money on the fact that nobody is going to look or think too hard about some contract that represents a tiny fraction of the years budget.
That still leaves the question of how one gets their foot in the door. Lots of us are aware of the budgets but we don't get how's sales work at that level.
That's what it means to be a "people person" in the context of trying to sell a product, yes. Getting within 2 degrees of a decision maker can open up millions for you, while being a rounding error for every company you work with.
I'd really like to know how he won contracts, just in general. Did he have some connections. And he doesn't even know how to get it to run on a server by himself? There's millions of people that can do that, if he can win contracts why worry about vibe coding at all, just hire someone to do it. Winning contracts is the challenge in my view.
Somewhere in the future, the new software engineering will be primarily about principles to avoid this in the first...
It's really nowhere near as complicated as making distributed systems reliable. It's really quite simple: read a fucking book.
Well, actually read a lot of books. And write a lot of software. And read a lot of software. And do your goddamn job, engineer. Be honest about what you know, what you know you don't know, and what you urgently need to find out next.
There is no magic. Hard work is hard. If you don't like it get the fuck out of this profession and find a different one to ruin.
We all need to get a hell of a lot more hostile and unwelcoming towards these lazy assholes.
This might not pan out to be the glorious victory of human craft as you’re imagining it to be.
Here’s a slightly different future - these AI rescue consultants are bots too, just trained for this purpose.
Plausible?
I have already experienced claude 4.7 handle pretty complex refactors without issues. Scale and correctness aren’t even 1% of the issue it was last year. You just have to get the high level design right, or explicitly ask it critique your design before building it.
This. I have this buddy, who is not an idiot by stretch of the imagination and more adventurous than me in some ways ( I don't really run agents on my machine ), but when I was looking at his prompts, I sometimes question how he gets anything done at all. It is vague and angry demands.
Not sure about the angry part, but vague sometimes works really good. The important part is to have enough good context pushed into the context window beforehand (codebase explorations, docs, etc). Then a vague prompt of the general direction gives the autocomplete more “freedom” to figure out the “best” approach given the context.
Doesn’t work well ofc in a one shot situation with no context.
Yeah, sorry, I myself was being vague, because I don't want to give any identifying info even by mistake. You are right; generalizations here are not as useful. I was talking about something the lines of 'can you make it better', but without llm having the context to understand what better could potentially mean. For brainstorming sessions, I love to start broad. Admittedly, I have limited experience with agents ( though current project intends to bridge that gap ) so it is possible I am missing something ( plus, to your point, I don't know his full setup ).
Maybe in the future but certainly no evidence of this anytime soon
Here's some anecdotal evidence from me - I cleaned up multiple GPT 4.x era vibecoded projects recently with the latest claude model and integrated one of those into a fairly large open source codebase.
This is something AI completely failed at last year.
Maybe you should try something like this or listen to success stories before claiming 'certainly no evidence' in future?
I'm no expert, but the skeptic's opinion I've heard would be to ask:
What evidence is there that we're not at or close to a plateau of what LLMs are capable of? How do you know the growth rate from 2023 to present will continue into 2029? eg. Is it more training data? More GPUs? What if we're kind of reaching the limits of those things already?
Since we're not experts, we treat it as a black box. What are the results? Is the quality of the results improving? Is the improvement accelerating or decelerating?
And the answer appears to be that the improvement is accelerating. So how could it be stopping?
I don’t think improvement is accelerating. We went from “computers can’t do these things at all” to “now they can” in a few years with the discovery of transformers, and now we get “it can do the same things, except incrementally better, at a drastically higher cost” every few months.
I don’t think that the current AI paradigm has infinite headroom for improvement, similar to how every other AI approach before it eventually hit a limit.
Ultimately, you are describing a fundamental problem with induction -- Hume's problem of induction to be specific. How can we know that anything that has been shown empirically in the past will continue to be true - we can't. Best to investigate mechanistically:
I don't see why we would assume that we are at a plateau for RL. In many other settings, Go for instance, RL continues to scale until you reach compute limits. Some things are more easily RL'd than others, but ultimately this largely unlocks data. We are not yet compute/energy/physical world constrained. I think you would start observing clear changes in the world around you before that becomes a true bottleneck. Regardless, currently the vast majority of compute is used for inference not training so the compute overhang is large.
Assuming that we plateau at {insert current moment} seems wishful and I've already had this conversation any number of times on this exact forum at every level of capability [3.5, 4, o1, o3, 4.6/5.5, mythos] from Nov 2022 onwards.
I think we're close to the plateau of what LLMs can do, but they will keep improving. IMHO the results are already showing diminishing returns.
The (leading) LLMs work by consensus, like Wikipedia, Openstreetmap, web search engine or opensource movement.
What I mean is if I ask LLM "create a linked list", its understanding (of what I want) is already close to the expected ideal. Just like Wikipedia article on linked list, for example.
But the LLMs will continue to improve in breath and depth of understanding the world, although technically (what they CAN do) they probably already peaked. Similarly, OSS movement technically peaked in the 90s with the creation of compiler, operating system and a database; doesn't mean that new opensource isn't being created.
There is so much money at stake, and so much money pouring into AI development, that I think we are going to continue to see gains for a while. People keep coming up with new agent harness techniques like chain of thought, tool calling, and memories. And then the big LLM companies figure out how to actually train their models to optimize the use of those techniques. To claim that we are reaching the top of the plateau is to claim that we are out of effective ideas for improvement. I think that's a ridiculous claim, the technology is too new. And because of the strong incentives to keep making these things better, it's pretty much a given that people will continue to explore ideas until we really are out of effective ideas. I don't think anyone apart from professional AI researchers have any idea where this is all going to settle.
I have personally had success telling Claude that some AI-written system is too complicated and ask it to rewrite it in a more logical way. This sometimes results in thousands of lines of code being deleted. I give an instruction like that if I see certain red flags, eg:
1) same business logic implemented in two different places, with extra code to sync between them
2) fixing apparently simple bugs results in lots of new code being written
It’s a sign I need to at least temporarily dedicate more effort to overseeing work in that area.
I somewhat agree with the AI psychosis framing of the OP. It takes some taste and discipline to avoid letting things dissolve into complete slop.
AI is currently, actively getting better. That might stop in a year, and you can argue whether it’s linear or exponential, but it’s very difficult to argue that it won’t get better in the near term. On the other hand, arguing that we hit a plateau at this time is just ignoring reality.
There are untold billions of dollars to be had if you can make this future come to pass. You don't need AGI to make it happen either. You just need to keep making the context windows bigger and keep coming up with updated training data. It's not the outcome I want, but it really does feel within reach. The only limiting factor is going to be token count and cost to process/generate those tokens. But if you don't particularly care about quality, costs are going to have to go up by several orders of magnitude before you start to regret firing your software engineers.
I don't know what happens in a decade when there are no junior engineers, skilled senior engineers are becoming rare, and the only data left the train LLMs on is 200th-generation slop. But AI slop being qualitatively slop is not enough of a obstacle to prevent that future from coming to pass. And billions of dollars will be "saved" along the way.
Companies are already putting billions out there just to secure and produce training data. And that's the isseue; spending X billions to make X-Y) billions isn't a profit, it's a gamble hoping Y becomes negative (or at least close to zero with a commodity that is profitable) . Real profits have not been made directly from the work on AI as of now. It's made from marketing a narrative of AI working.
That's what makes this whole house of cards dangerous. The prescription to psychosis is profitable. Aka, selling a grift.
With GPT 5.4 or 5.5 I did not notice degradation in performance when it was working on a large 5k line file containing a WebView, JS scripts, as well as native UI.
I instructed it to split it up anyway, yet I wonder how often the concerns around the mess are imaginative rather than practical.
I think AI rescue consulting is going to be come a significant mode of high value consulting
I thought the same when I saw development outsourced to Indians that struggled to write a for loop.
I was wrong.
It turns out that customers will keep doubling down on mistakes until they’re out of funds, and then they’ll hire the cheapest consultants they can find to fix the mess with whatever spare change they can find under the couch cushions.
Source: being called in with a one week time budget to fix a mess built up over years and millions of dollars.
The complexity you would come to the rescue to solve, would that be from AI or from the style of programming you let the AI have? I mean, you have very different problems if you use functional style vs object-oriented. It is up to the programmer to realize they want a functional style and request that from the AI, as much as possible. Even AI cannot imagine every state transition, unless it is so smart that it should be the one telling you what to do.
Heh. Got a customer recently around this. Entire infrastructure and CI/CD vibecoded. They half implemented Kubernetes in Github Actions that were several thousand lines long and impossible to understand.
I think the problem will get worst. I dislike the marketing around AI, but I do think it is a useful tool to help those who have experience move faster. If you are not an expert, AI seems to create a complex solution to whatever it is you were trying to do.
If you are not an expert, AI seems to create a complex solution to whatever it is you were trying to do.
I've been watching non-developers vibe code stuff, and the general failure mode seems to be ignorance of 3-pick-2 tradeoffs.
They'll spam "make it more reliable" or some such, and AI will best-effort add more intermediary redis caches or similar patterns.
But because the vibe coders don't actually know what a redis cache is or how it works, they'll never make the architectural trade-offs to truly fix things.
I’ve noticed something similar with vibecoded game rendering logic submitted by peers. Sometimes it will be peppered with extraneous checks for nullptr, or early returns on textures that have zero size.
I often wonder if it’s the statistical nature of the LLM mixed with a request in the prompt.
AI LOVES defensive coding. I asked you for code to filter and reduce an array. I didn't ask you for a method that makes sure the array exists and is an array before it does anything else.
I'm with you on this one, having "vibe coded" some smaller internal tools on GPT 5, and then re-vibed it on Opus 4.6 and 5.5 -- they basically just fixed all of the problems without me doing much of anything other than prompting it to look at the existing code and make it "better".
Pretty much. We're intensely vibe coding something that has gone through so many requirement changes. The code has become very gnarly. I took a stab at basically one prompt rewrite of the whole thing. And it wasn't there, but it was 80% of the way there. and a hell of a lot cleaner.
Are you sure about this? Yes, there is a stable set, but they are used in all of the wrong places, particularly in places where they don't belong because juniors and now AIs can recite them and want to use them everywhere. That's not even discussing whether the stable set itself is correct or not - it's dubious at this point.
Those design principles it will take us 20 years to learn are just the principles for writing good maintainable, debug-able, understandable code today. Will just take 20 years to figure out they still apply when AI writes the code, too.
Many teams did this before AI too. They start faster and end up with hard to refactor or extend code. For example, think of teams that don’t version their API /api/v1, that blocks a whole category of refactoring and extensions. Or teams that have random state transformation routes instead of following restful actions.
Interesting perspective. Fundamentally at conflict with the data, science, and 20+ year trends of AI coding systems - to the point of dogmatism. But interesting from a sociological point of view.
is this true because training companies have not been training AI for both performance and brevity (or some other metric like that)? If this becomes a much more serious issue surely they would adjust the training processes
What you're describing really isn't a new problem for organizations. Historically it's been a team of humans not using AI who gets over their skis and they have to have other more capable humans (also not using AI) to bail them out.
I don't understand this point of view at all. There's a symmetry that is going entirely unappreciated by most of the comments in the thread: just as I can give Claude X,000 words of text to use to describe the code I want it to write, I can also give it some existing code and ask for X,000 words of text explaining what it does. (Call it, oh, I don't know, a "spec," maybe.)
The explanation, in turn, can be fed back to recreate the functionality of the original code.
At that point, why care about the code at all? If it works, it works. If it doesn't, tell the model to fix it. You did ask for tests, right?
That is where we're indisputably headed. It's not quite a lossless loop yet, but those who say it won't or can't happen bear a heavy burden of proof.
Code is not spec. There is an implementation spectrum.
On one end, you have code that can perform only the behaviour explicitly declared in the spec, but has to be thrown away and rewritten for any new or updated spec.
On the other end, you have code that implements or anticipates a wide range of future possible specs including the given one.
The AI can operate on any point on this spectrum, but it's not very good at choosing. The more complex the software, the more such choices need to be made.
When the number of bad choices reaches a certain critical mass, even a skilled engineer becomes powerless to undo all the bad choices, and even a powerful model becomes unable to reduce it back to a coherent spec.
Some people are mindful about what they get and don't get from amazon and don't die from prosperity. ("you might use AI to increase your prosperity")
the rest of the world eats too much and dies of heart disease/diabetes. ("the rest of the world will flounder more and AI will do more stuff to them than for them")
I've already done a handful of these gigs for early vibecoded products that had collapsed in on themselves. The scope of work was to stabilize the product and only make existing features work.
The issues have all been structural, not local. It's easier to treat it like a rewrite using the original as a super detailed product spec. Working on the existing codebase works, but you have to aggressively modularize everything anyway to untangle it rather than attack it from the top down.
All of these projects have gone well, but I haven't run into a case where a feature they thought was implemented isn't possible. That will happen eventually.
It's honestly good, quick work as a contractor. But I do hope they invest in building expertise from that point rather than treating it like a stable base to continue vibecoding on.
I've worked with many people over the years. A bunch of product people have struck out to make their own thing now that they can get a feedback loop going. I just keep in touch with people. They know my services are available, so if they have a need they reach out.
The greatest asset in this type of work is genuinely liking people, being good at what you do, and keeping in touch. My email is easily findable for a reason.
So, there is no secret sauce? Just work for people and have a hand at their contact info?
The one part I do wonder is how to "keep in touch". Maybe it's a generational thing as a young millennial (some would call it "Zillenial") but the biggest issue in my networking over the year (cough and the dating scene cough) is ghosting. You think you hit it off, try to follow up the day after, and proceed to never again hear from them.
Purely AI written systems will scale to a point of complexity that no human can ever understand
But won’t those more complex systems presumably solve more complex problems than the systems that humans could build? Or within a comparable time?
I think it is reasonably safe to assume at this point in the game that these AI systems are increasingly able to reason rigorously about novel problems presented to them, of ever increasing complexity and sophistication.
That sounds so horrible, though. It's akin to people working as COBOL devs because someone has to do it, so they'll get the big bucks. Except I've never heard of anyone who actually likes COBOL and the more I've learned about how mainframe development actually works, the more horrified I've become haha. Dealing with an LLM spaghetti codebase sounds like hell.
My company and my buddy's company, we're experiencing the same thing. We are trying to fire a SAAS vendor and it's become the hot new project. Now we to these meetings with 50 different people that are allegedly stakeholders, two or three product managers who have already vibcoded their version of something.
Ultimately, if you want to move fast, it's better just to have one engineer vibe coding something. but, that engineer is under so much pressure. Now he's got a legacy mode and another legacy mode because the requirements keep changing. And now there's a deadline in four weeks.
This all could work just fine, but the ungodly amount of attention that this world is getting puts too many cooks in the kitchen, which is always a recipe for disaster.
Sounds like wishful thinking by a human programmer. Probably more likely tools will be rewritten from requirements or the old code will be refactored by newer models.
I shut down AI Agent fanatics on the regular. But chop one head off there and two take its place. And I say that as someone working with Claude and Codex daily. While they are both incredibly good at clearly described and defined atomic tasks, application scope makes them lose their minds and the slop ensues.
The race to invent variants of Gas Towns, Ralph loops, pump out videos, blogs, etc. showing off greenfield development with cleverly named agents running in parallel is another case of engineering people diving head first into Resume Driven Development.
Sure there are industry changing things going on. What if you're working on an app thats a decade old and has had different teams of people, styles, frameworks (thanks to the JS-framework-a-week Resume Driven Development)? Some markdown docs and a loop of agents isn't going to help when humans have trouble understanding what the app does.
I work for a small telecom services provider whose current VP immediately set an AI course when stepping on board 6 months ago. Involving AI in everything and every task is now our first priority - across all employee segments, not just us system developers - and leadership is embarking on a program to measure employees' AI usage levels as a means to gauge everyone's individual efficiency. It's like the era of the evangelic crypto bros all over again.
There's a lot of people writing bad code. With AI being forced top down (with the promise of turning people into 10x-ers), we're going to get a lot of people writing bad code 10x faster.
I really do worry - I especially worry about security. You thought supply chain security management was an impossible task with NPM? Let me introduce to AI - you can look forward to the days of AI poisoning where AIs will infiltrate, exfiltrate, or just destroy and there's no way of stopping it because you cannot examine the internals of the system.
AI has turbo charged people's lax attitude to security.
Not security, but I ran into a related supply-chain issue recently. I needed a library to perform a moderately complex task, and found one in the ecosystem I was working with that had been around for a while, appeared reputable, and passed my cursory inspection. So I dropped it in, got the feature implemented, and moved on.
Some time down the line, I discover CPU being maxed out, which is showing up in degraded performance in other parts of the system. I investigate, and I trace the issue to a boneheaded busy loop in this library that no human with the domain expertise to implement the library would have written. Turns out I'd missed one deeply-buried mention in the README that maintenance was being done via AI now, and basically the whole library had been rewritten from the ground up from the reliable tool it used to be to a vibecoded imitation.
Yeah, yeah, sure, bad libraries existed before all this. But there used to be signals you picked up on to filter the gold from the dreck. Those signals don't work anymore.
I don't think it's helpful to call this psychosis. N
Beyond that I don't think it's even irrational.
It is definitely factual that there is a complete paradigm shift in the prioritization of quality in software. It's beyond just AI side effects, and now its own stand alone thing.
There have always been many industries, companies, and products who are low on quality scale but so cheap that it makes good business sense, both for the producer and the consumer.
Definitely many companies are explicitly chosing this business strategy. Definitely also many companies that don't actually realize they are implicitly doing this.
Wether the market will accept the new software quality paradigm or not remains an open question.
The hype or psychosis is mainly by mediocre/non expert/middle manager/you name it, especially when a person who never wrote a single line of code suddenly is making a wall of text, and it actually works!? Oh my!!
But in reality, anyone who knows their field and are going after certain specific issue, they will find soon how AI is nothing but an assistant, sure it can help and automate some stuff, but that’s it, you need to keep it leashed and laser focused on that specific issue. I personally tried all high end ones, and I found a common theme, they are designed to find a solution or an answer no matter what, even if that solution is a workaround built on top of workarounds, it’s like welding all sort of connections between A and B resulting in a fractal structure rather than just finding a straight path, if you keep it going and flowing on its own, the results are convoluted and way over complicated, and not the good complexity, the bad kind.
The entire problem is vibe coding is only good for demos, prototyping and finding signs of product market fit without actually releasing a product into the market.
You should not release a product into the market unless you have a good enough product that can keep you and your client compliant, safe and secure - including not leaking their customer info all over the place.
Prompt injection risk, etc. are massive for agentic AI without deterministic guardrails that actually work in practice.
Stop testing in production if you're shipping in a regulated industry. Ridic!
If you're not technical, you can get someone who is after signs of p-m fit, demos, but BEFORE deployment. This is common sense and best practices but startup bros dgaf because they're just good at sales and marketing & short term greedy.
I saw this first hand at a company, and I think this is what happens when you combine FOMO with an utter lack of industry best practices. No one knows where they are going, but are convinced they are not getting there fast enough.
What's more, the only people they talk to about it are others at the same company. There is no external touchstone. There are power dynamics from hierarchy. No new ideas other than what is generated within the company. In other circumstances, this is a textbook environment for radicalization.
I would encourage all leadership to take a deep breath. You have time to think slow.
Totally unrelated pet peeve of mine, I hate when people write this: "MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery)".
You first use the full words and then introduce the acronym that you're going to use in the rest of the text: "Mean Time Between Failures (MTBF) vs. Mean Time to Recovery (MTTR)".
With the latter, readers understand the term immediately, even if they don’t know the acronym. And they don't have to read these weird letters before getting the explanation.
Amazing how the dev community is suffering from a similar inability to approach the subject of real world AI efficiencies and business benefits. I don’t think it’s helpful to accuse the other side of psychosis. It disqualifies any data or experience they bring to the conversation.
It seems the diagnosis of psychosis is too quick: it seeks to reestablish the frame of expert for the developer identity that is being replaced by it.
“It feels like entire companies are deluded into thinking they don’t need me, but they still need me. Help!”
The broad sentiment across statements of this “AI psychosis” type is clear, but I think the baseline reality is simpler. How can you be so certain it’s psychosis if you don’t know what will unfold? Might reaching for the premature certainty of making others wrong, satisfying that it might be to the ego, be simply a way to compensate the challenges of a changing work environment, and a substitute for actually considering the practical ways you could adapt to that? Might it not be more helpful and profitable to consider “how can I build windmills, ride this wave, and adapt to the changing market under this revolution” than soothing myself with the delusion that all these companies think they don’t need me now, but they’ll be sorry.
The developer role is changing, but it doesn’t have to be an existential crisis. Even though it may feel that way — but probably it’s gonna feel more that way the more you remain stuck in old patterns and over-certainty about how things are doesn’t help, (tho it may feel good). This is the time to be observant and curious and get ready to update your perspective.
You may hide from this broad take (that AI psychosis statements are cope) by retreating into specific nuance: “I didn’t mean it that way, you’re wrong. This is still valid.” But the vocabulary betrays motive. Resorting to clinical derogatory language like “AI psychosis” invokes a “superior expert judgment” frame immediately, and in zeitgeist context this is a big tell. It signifies a need to be right, anda deeply defensive pose rather than a clear assay of what’s real in a rapidly changing world. The anxiety driving the language speaks far louder than any technical pedantry used to justify it, and is the most important and IMO profitable thing to address.
I feel in a really weird position where I both really dislike what AI is doing to the experience and practice of writing code, to the point where I want a job doing literally anything else besides using the computer, but also think that these tools are extremely powerful and only getting better.
I think Mitchell's point is well taken -- it's possible for these tools to introduce rotten foundations that will only be found out later when the whole structure collapsed. I don't want to be in the position of being on the hook when that happens and not having the deep understanding of the code base that I used to.
But humans have introduced subtle yet catastrophic bugs into code forever too... A lot of this feels like an open empirical question. Will we see many systems collapse in horrifying ways that they uniquely didn't before? Maybe some, but will we also not learn that we need to shift more to specification and validation? Idk, it just seems to me like this style of building systems is inevitable even as there may be some bumps along the way.
I feel like many in the anti camp have their own kind of reactionary psychosis. I want nothing to do with AI but I also can't deny my experience of using these tools. I wish there were more venues for this kind of realist but negative discussion of AI. Mitchell is a great dev for this reason.
I've never had more fun coding, but the key is actually still writing the code yourself. The LLM has terrible judgment but an encyclopedic knowledge and the ability to pick out important details in a sea of information. Their worse use is producing code, but somehow that gets all the energy. Being an LLM babysitter is energy draining and you feel less and less in control. No job is worth being miserable doing something that you used to enjoy.
I've used both ChatGPT and Claude, they seem interchangeable for my needs. I only use the web prompt interface except for the rare occasion that it is helpful for it to have the context of my entire project. I think less is more when it comes to LLM interaction, but sometimes they are exactly the right tool for the job.
I didn't realize you wanted that information too, I could probably bore someone to death talking about it.
Planning: I often ask it to help me plan an approach if we are dealing with something I don't have a lot of experience with, most recently working with the DOM. If there is a library or an API that is new to me, I ask for an overview and run my plan by it for comments. Feed it the documentation and it is like talking to author.
Coding: I have a pretty reliable sense for when a section of code that I want to write is obvious enough for the LLM to one-shot based on the other code in the file, and on those occasions I call in completion. I do this with code that I can verify at a glance.
Analysis: If I have any uncertainty at all about the code I've written, I run it by the LLM to find issues. Out of all the other uses, I think this is the most productive and time saving. If I run into a bug and I'm stumped, I show it the section of code. I'm amazed at how good it is at finding mistakes.
I'm working solo as a full stack developer coming from a different background, so I sometimes find myself out of my depth. Having access to the breadth of knowledge that an LLM brings and its attention to detail has been game changing. I've tried a couple agents and configuring them to work competently seems like a rabbit hole, and I like the tight control over the context that chatting with the web prompt interface brings. It seems like half the value is putting into words my intent, it forces me to have a cohesive understanding myself. It is like rubber duck debugging where the duck can actually talk back and sometimes provide the critical part that I'm missing. I have it speak like a pirate which is just for fun but sometimes the sailing metaphors that it uses are really intuitive.
I've been working on a C++ backend for F# and while I'm very familiar with F# and it's AST I barely know C++. The amount of time I save being able to ask things, check my understanding, get design patterns, and paste issues I'm having for a fix is insane
I ran into an issue where I was getting a segfault and everything looked right in the debuggr, including expected values near the segfault. Turns out I wasn't using placement new somewhere I needed, and the data for the object was getting copied but not the vtables. I have no idea how long it would have taken me to figure that out on my own because the segfault was coming from so far away
I haven't had the opportunity to use LLMs much for coding since I'm not working right now, but I can second how much of a boost just getting specific answers to my questions instead of reading tons of whatever online searches return is.
Rubber duck that talks back is a nice way to put it
This is more or less how I use Claude and Kim 2.5 via Kagi. If I just let it spew out code I have no idea what it does and no interest whatsoever to try to comn through it all. But when I have need to ask about syntax or correct use of library function etc. - I’m learning C++ - and can’t grok the docs, it can be incredibly helpful. Also is great at finding bugs.
I think of it kinda “very knowledgeable dumb person” - it knows everything but understands fuck all (although it can appear to do so just by breadth of information it has). If I can formulate a question in a way it gives me the correct info it helps me to conceptually understand the problem better then filling out the blanks. Often I figure out the answer to my question just by writing it down without needing to prompt it, so speaking rubber duck is very apt way to call it.
working in a large codebase I use Claude for code understanding and the code reviews from Macroscope have caught bugs for me a bunch of times. Usually if I use claude it’s for refactoring a and source to source transformations that would be too confusing for me to figure out how to do with e.g. ast-grep, but that I can prompt in a minute or two and then have claude work through it. It’s stuff I could do without LLMs but it’s less effort to use them. I don’t let it write new code, because it decays the process of programming as theory building.
not the person you're replying to, but someone who agrees with the gist of their message - I personally use Claude Code as a better Google search for debugging and syntax.
It used to be "oh, why am I getting an error on line 352, let me google the error message and wade through Stack Overflow answers" now it's "Claude, why am I getting an error on line 352? Ah, it's because $REASON, let's see if that fixes it, yes, thank you."
Obviously reading the official documentation is very useful, but sometimes you can't find anything that relevant to your exact use case, and forums are also very useful, but it can take hours or even days to get a reply to question when the LLM can do it in like a minute.
In my experience, its the opposite. The AI is very good at writing code, but it is unreliable at any kind of design. I use it as a fancy form of autocomplete. I give the broad strokes: "add a method here and change all but this one caller to use the new method", "Apply this design pattern here for this change but don't do this other thing". It completes the task reasonably well and sometimes even remembers to run the code formatter and check that tests pass.
If I ask it to me produce a design, I'll almost always end up with something unworkable or inefficient.
Though if you push it hard enough then it can sometimes give you a good description of what existing code does and how it does it (which can be easily verified).
Can you pro-AI people at least agree on what is it even useful for?
Every thread is endless back-and-forth between the "AI works great and vibe coding is the future" and "no, AI works great as long as you don't vibe code" camps.
AI is in like year 4 of being useful for the task. And it’s changing rapidly. We don’t know what it’s good for yet!
But pick a technology used to write code and you’ll see many of the same things. Broad unit test adoption happened more then 20 years ago and people still can’t decide what it’s good for or even if it is useful.
That’s because it depends on the circumstances of the problem and the person solving it.
So no, we can’t decide exactly what it’s good for and if we could there would be exceptions. But that’s not an indictment of any tool.
Is it so hard to imagine that people who find LLMs useful might have different perspectives on it? If you put a lot of energy into being against something you might think that everyone who doesn't agree puts the same energy into being in favor of it, but that's not real life.
Can developers agree on what the best programming language is? The best framework? The best OS?
AI is a diverse technology and will work differently for everyone, just like JS is a really, really bad technology for many, yet widely used.
On top of that it (as coding agent) is brand new, just a few years old. Looking at the history of technology it will probably take a few decades for it to settle into some sort of stability. Which does not mean it does not provide value during this time of exploration, however there is no trodden path leading you there.
Fancy autocomplete is a solid use. I use the same pattern replication technique whenever the situation allows for it. It's great for following your lead, but unreliable at taking the lead. Even if it was more reliable, I won't have the same working knowledge of the structure it created.
My biggest complaint about AI is getting it to lock onto current or specific information has been darn near impossible. Its definitely there in its training data but for the life of me I cannot get it to stop bleeding long outdated or external information into its responses.
The LLM has terrible judgment but an encyclopedic knowledge
Bingo. Claude can bang out nearly correct code when I give it an idea, but it doesn't have the idea and repeatedly misjudges both how much work remains and what kind.
On the other hand, I don't know all the ins or outs of macro expansion in yaml at compile time or when and where macros run, enabling us to conaume their results elsewhere in the yaml. Frankly, if I had time, I'd happily spend time on that and learn more about it. I don't, though. Claude knows and does the guessing and checking. So I provide the concept and it translates into a horrible soup of yaml. Clearly I'm able to press forward with ignorance, which is dangerous. There's a real risk that I'll wind up with the kinds of unhealthy work that worries the author of the tweet.
After turning off my brain a few times and ending up in a place I don't want real fast, I am learning to ride this dragon.
And, you are right - use it as a fast typer, not a fast thinkier.
And for those who claim that AI is a good code writer - wrong. It can OUTPUT a wall of code, but it's overeager to flood you will legacy code on arrival to "solve a problem". It's harder to write LESS code, which is still the goal (even more true today).
But humans have introduced subtle yet catastrophic bugs into code forever
So now the AIs will do more of that, at superhuman speed.
will we also not learn that we need to shift more to specification and validation
We'll just quickly learn what we've been trying to do for decades, while also treading water in floods of more code than has ever been written before? And some of the motivations to write correct code are being deflated - "just vibecode it again and see if the bugs disappear, it only took a week and $200."
I think the commenter was referring to formal verification with "specification and validation": have the LLM emit formal proofs about invariants etc.
Currently the bugs are found by people using LLM's but aren't the developers. As more projects start getting access to compute, they can run those LLM searches for bugs themselves, and can simply prevent shipping the bugs.
I'm surprised no one has tried making any statistical analysis of bug densities, and "bug authors" in an attempt to identify untrustworthy developers, regardless of intent. Given a dataset of authors and prior bugs, it may help find more bugs by tracking their pull requests with higher scrutiny...
Some people may end up with an eternal stain if they've been taking money to submit vulnerable code to code bases...
I feel like many in the anti camp have their own kind of reactionary psychosis.
You're using psychosis wrong. My literal reality is my entire industry trying to use Ai as an excuse to payoff hundreds of thousands, to millions of American engineers in lieu of outsourcing work overseas. It's having hostile promots to use AI that never truly go away (if you're even given an option to turn off the prompt). It's seeing an emerging generation completely stunted because AI's best use is to cheat the education system and ruin the youth's critical thinking. It's looking in apallment at proposals for data centers that take more energy than the state actually has.
And while you can try to call these exaggerations, you're falling into the very psychosis of this article if you want to deny this reality as a whole. "but the tech is making us so productive" is not a valid justification to literally collapse human society as we know it.
Not FANG, but I work at a company that operates some infrastructure at scale. What I've seen is after we've rotated through a number of different tools, in different pilot groups, eventually converging on tool X (a custom, internal wrapper around opencode).
Now every "working session" like meeting, at the team or dept level, has been around how to use tool X. Tricks using tool X. Problems using tool X. I can't help feeling if we had spent the same amount of time building up core knowledge/contempencies around say, design patterns, networking, specs, we'd be in a better place for building.
Instead we are going to have a few thousand people who know a tool really well.
Maybe this is what will turn software engineering into an Engineering field.
Right know, prompters are setting up whole company infrastructure. I personally know one. He migrated the companies database to a newer Postgres version. He was successful in the end, but I was gnawing my teeth when he described every step of the process.
It sounded like "And then, I poured gasoline on the servers while smoking a cigarette. But don't worry, I found a fire extinguisher in the basement. The gauge says it's empty, but I can still hear some liquid when I shake it..."
If he leaves the company, they will need an even more confident prompter to maintain their DB infrastructure.
Maybe this is what will turn software engineering into an Engineering field.
Oh man, I think you may have touched the third rail here.
My first job out of high school was as an AutoCAD/network admin at a large Civil & Structural firm. I later got further into tech, but after my initial experience with real Engineering, "software engineering" always made my eyes roll. Without real enforced standards, without consequences, it's been vibe engineering the whole time.
In Civil, Structural, and many other fields, Engineers have a path to Professional Engineer. That PE stamp means that you suffer actual legal consequences if you are found guilty of gross negligence in your field. This is why Engineering firms are a collective of actual Professional Engineer partners, and not your average corporate structure.
The issue is that in software dev, we move fast, SOC2 is screenshot theater, and actual Engineering would slow things way down. But, now that coding is fast, maybe you are correct! Maybe vibe coding is the forcing function for actual Software Engineering!
___
edit: I just searched to see if my comment was correct, and it turns out that Software PE was attempted! It was discontinued due to low participation.
NCEES will discontinue the Principles and Practice of Engineering (PE) Software Engineering exam after the April 2019 exam administration. Since the original offering in 2013, the exam has been administered five times, with a total population of 81 candidates.
Note that other types of engineering are also often vibes based. The mechanical engineering for a rocket engine is extremely rigorous but the engineering for an injection molded housing for a cheap cell phone is a lot more about following a few heuristics and getting it out the door. Even in robotics where I work, it’s mostly about making parts that pass whatever acceptance tests you come up with. In civil engineering and aerospace failure costs human lives and millions or billions of dollars. In robotics maybe you have some machines fail in the field but in many instances you have one overarching safety system and many of the parts are irrelevant to that. The camera housing for example. So no paper trail or mathematical design validation is required to prove you designed it right. Often those are desirable but if you just manufacture it and test it a lot you’re probably fine.
This was something I noticed in my early career in mechanical engineering and later doing PCB design and software for robotics. It’s easy to find firms that just need adequate parts without the professional certifications or ass-covering calculations of other engineering fields.
All this to say, it’s not just software versus the rest of them. From my position, civil and aerospace seemed more like the exception while much of the rest of the engineering world is more vibes based.
What makes it a profession is not just the certification, it's the burden of responsibility for consequences. Your lawyer, accountant, and real engineers carry "we need insurance for this" level of risk in their work, all the way up to "can go to prison for getting things really wrong".
Until and unless software is held to that standard, software will never be engineering and always just a craft that can be performed to any or no standard.
Eh writing software for healthcare, or aircraft or self driving cars is more rigorous than an EE working on industrial lighting or toys.
Im sure for the most part, engineers in physical space deal with the same kind of tradeoffs software engineers make, where you try your best based on industry standards, personal past experiences without some way to prove what youve done is right
Eh writing software for healthcare, or aircraft or self driving cars is more rigorous than an EE working on industrial lighting or toys.
That’s a relatively small field within the software industry.
Most of the work being done (adding new fields to CRUD apps etc) is glorified clerical work, where the people doing it are rightfully fearful of being automated out of existence by AI.
Now imagine if you’re one step removed. You don’t see the cigarettes, smell the gasoline, nor see the fire extinguisher gauge. You only see the servers running business-as-usual. Those “engineering” guys are always drama queens, you think. We have processes and fire extinguishers when shit hits the fan, right?
That’s basically every M2, and many if not most M1s, in the last 10 years. So fuck it. Why does any of it matters?
Yes, you can’t blame AI for it. But it was a self limiting system. You couldn’t just go to a fresh college hire and ask them to do deep surgery on the entire stack. You would go to your very senior engineers for that. Those senior engineers will push back on some stuff then in the push-pull cycle you would have to settle at some middle ground.
With AI I’m seeing managers literally get an intern, ask them if they can change fundamental assumptions of a system, give the intern claude 1M window, have the intern ready with a 37k line PR in an afternoon and then go ping a senior engineer if they can “take a look”.
As a junior dev there is this pressure to produce code, add features, and investigate bugs within unprecedented time period. I know whole code base is fking up but i will still add that feature or do a sloppy bug fix without digging deeper.
In my experience, AI really lowered the bar for bad code in the name of delivering faster.
I have seen people write highly complex code where all the complexity was not necessary. Think: deep unnecessary branching, pointless error handling and retries which make no sense in our context, hand-coded parsing using regexps, haphazard data flow, functions which seem purely computational but slyly make API calls, pointlessly nullable model fields, verbose doc comments which describe the implementation instead of the contract. I could go on.
The worst part is, even when "prompted" by bad coders, it works in the end. Even has tests (ostensibly mock-ridden, a pet peeve of mine which always falls on deaf ears). So I cannot reject the PR without being an asshole.
I am no luddite. I make heavy use of AI, with all the skills / AGENTS.md / style guides and clear specs, then review every line of code, prefer testing with minimal mocking. I'd even say with right prompting, it can write better low level code than me (eg: anticipating common error conditions).
But my biggest fear about AI is how it enables normies with little to no understanding of CS principles to produce code faster which looks correct but slowly poisons the codebase.
I have a friend, smart guy, who is writing web services and “connecting them together” for a large firm; he has absolutely no programming experience.
Talking to him, he told me he couldn’t even reverse a string. He is at once many times more valuable than ever before to his company, but also far more dangerous than ever before.
This is what fascinates me. I have a friend, also a smart guy, who has made it to the point he’s at by being a kind of solutions expert. He’s an IT guy, basically. He’s very technical but has never claimed to be a software engineer. He’s writing software with Claude now. The other day he sent me a screenshot of some other team at his work asking him to shut off something he made that was brutalizing an API of theirs. I asked him if he had ever heard of a 429 or exponential back offs. He said no. How do you meta-prompt for that without knowledge?
You can create an agent in Claude with the role of Technical Lead / Architect and have it review your code. That depends on your agent specification. Just have ChatGPT generate that first.
If you get the logs you can feed them in and ask for improvements, that sometimes helps.
He's "smart" but he chooses to be in a business where he's presumptively willfully ignorant of the fundamentals (since he surely should be able to learn to reverse a string if he wanted to learn)? He doesn't have a more lucrative opportunity available? Or does he somehow have a skillset that makes him able to "connect web services together" by prompting AIs in ways that other people (including ones who can reverse strings, etc.) couldn't?
This form of being "smart" is a bit difficult for me to comprehend, I must admit.
This form of being "smart" is a bit difficult for me to comprehend, I must admit.
I strongly agree with this. Suddenly with the mass adoption of LLMs there are so many smart, yet naive people out there willing to toe the line. Why these smart people couldn't bring value in a million different other ways is, of course, left unsaid.
They're not even trying to dress up these bullshit stories anymore. In truth it doesn't matter if you believe it. So much buzz is people just talking to themselves out loud.
Well that’s not his primary job. It’s an extra task he’s doing at his job. At a non-tech company with small/horrific engineering components, someone in the business who can do any programming (or vibeing) is indistinguishable from magic.
When I read the discussions about AI making code worse I keep bringing the same argument: people made bad code even before AI. Average coder is barely functioning and that's a fact.
And we were safe from them because they couldn’t produce a mountain of code every day. But soon many places will be buried under a planet of unmaintainable code. It’s adding friction and operational cost and often not adding value.
People could, however, learn to not make bad code. LLMs are incapable of that feat because they do not have any understanding or ability to reason. They are strictly worse than a human.
As others have elaborated, the problem is empowering them to ship mountains of bad code;
And yeah, many semi-technical M2s or even M1s can't distinguish bad code from good code, or worse bad architecture from good; this is golden time for those who are willing to sacrifice the future for present. Just burnnn'em tokenzzz.
In my experience, AI really lowered the bar for bad code in the name of delivering faster.
I would've believed that 6 months ago, but not now.
If you have a good codebase with proper rails, hygiene and architecture, AI will produce better code than most engineers out there.
People forget that 90% of the field has always been charlatans barely able to implement a fizz buzz or go much beyond trial and error googling.
I'll say even more. I'm in the 10%, and it's increasingly clear to me that AI writes in minutes code that's better than mine.
Even stellar and respected OSS engineers are nowadays leveraging AI and guiding it less and less everyday beyond giving indications of what kind of data structure they may want for a complex problem or the kind of architecture they are looking for.
In any case, I don't like this field anymore, I have no joy from it, way too much work, way too many changes a human can cope with both on product and technological level (not even counting AI and its tooling itself). The interesting parts of thinking an entire afternoon or week experimenting to get that design right disassembling the pros and cons are gone.
Even if you want to do that, it's just faster to launch 6/7 worktrees with the different ideas and judge the results. But you don't get as intimate with the problem and the amount of information is way more than you can process.
Forgive my ignorance, but if the corpus of coding data was always 90% bad, isn't that the same data being used for training LLMs? How are they magically any better than that average?
I'm hand rolling a project right now because even frontier models I use bloat things beyond comprehension. Because I'm intimately familiar with the domain, I know the shape of things, how the data should flow, and so on, and if l even if I spec it clearly AI will write 2x to 5x the amount of code necessary to make something work.
"beyond comprehension" is a good way of putting it. I've been genuinely baffled by some of these AI designs - why any intelligent thing would write >10 lines of bloat for what should be a one-liner.
"beyond comprehension" is a good way of putting it. I've been genuinely baffled by some of these AI designs - why any intelligent thing would write >10 lines of bloat for what should be a one-liner.
As Anthropic's drones say: treat Claude as your genius coworker. Don't think yourself, don't judge, the machine must know better than you. It is the genius, after all, not you.
Proper rails, hygiene and architecture need to be actively maintained, they don’t just continue to exist in a developing codebase. Historically, a small proportion (the 10% as you say) had a disproportionate amount of influence on coding standards. When they can no longer keep up with that ongoing maintenance, which we’re seeing with the increased pressure to ship code, the hygiene will regress. We’re riding the tail of all the engineering practices we’ve developed as an industry.
This is what I’m seeing, anyways. Junior engineers are being rewarded for shipping so much code, it’s impossible to evaluate it all, and subtle changes in existing patterns are slipping through. Eventually all those subtle changes transform the rails.
So many fallbacks. So many function_exists. So much pointless type casting. I swear it’s like the system prompt is designed to waste as many tokens as possible.
it works in the end. Even has tests (ostensibly mock-ridden, a pet peeve of mine which always falls on deaf ears). So I cannot reject the PR without being an asshole.
This is a social problem that I had thought the industry had solved a long time ago.
I agree 100%. At work I'm teaching software engineering principles to system administrators and they, too, often use AI without thinking. Then it's on me to provide feedback on the PRs they barely read themselves. Not a lot of fun but slowly but surely they're learning.
The worst part is, even when "prompted" by bad coders, it works in the end. Even has tests (ostensibly mock-ridden, a pet peeve of mine which always falls on deaf ears). So I cannot reject the PR without being an asshole.
Yes, you absolutely can. And you should. Try to teach them lessons and what patterns to watch out for, then tell them to put those insights in their CLAUDE.md, so that their agent becomes better, too. You can also tell them to just copy your own CLAUDE.md, like I did: https://github.com/codethief/ENGINEERING_PRINCIPLES.md
Maybe this is what will turn software engineering into an Engineering field
I think it’ll be the opposite. Maybe it’ll be what will eventually cement the field as “talent” based field. Just like it was difficult to quantify what makes a flute player better than another, how good your are at endlessly prompting a blackbox machine would be the only measure. The engineers of ol’ whoe developed kernels and drivers would be thought of as the “crazy people who put the flute against their temple to tune it” LOL. we don’t need people like that. You can just buy a flute tuning device. who gives a fuck? Can you make the next “Shake it, Shake it”?
I work at software in a medical setting. We are piloting an integration with a startup for measuring [some bodily variable relevant in ICU setting]. They are obviously vibecoding (docs are telling) and their API is failing in unexpected ways that they are not able to resolve. I am just waiting when this are going to harm somebody.
This is the pattern you will see when medium-successful ignorant people take o ver a system that was based on some kind of standard.
You can see the same approach is taken by Trump and other people.
“You have TDS!! He is actually doing good. He doesn’t follow rules because the system is rigged etc.”
These arguments border on religion because it is predicated on you believing their ignorant point of view in the first place.
Engineering and science is built on rigor and empirical evidence, it is not built by scammers/businessman/ignorant-people/politicians because that is just not how it works
This reminds me of Rich Hickey’s “Simple Made Easy” and his approach in making Clojure.
Even before LLMs generating entire programs, complex frameworks allowed developers to write the initial versions of programs very quickly, but at the cost of being hard to understand and thus hard to debug or modify.
Some of us are betting that the AIs will always be smart enough to debug, maintain and modify the programs written by AI, no matter how convoluted or complex. I’m not so sure.
That people don't realize full test coverage just means every line is hit, not that everything is correct is always funny to me. (I don't view as an argument against tests, but with AI it's especially important as if you're aren't careful it'll be very happy to make coverage that is not quite right.)
Just talked to an exec yesterday about their multinational company, where the newly-installed CEO just came in with "everyone needs to be using AI" and "we should be doing everything with AI".
I cautioned them that this a terrible idea -- you have business people who don't know what they're talking about, and all they know if "if we don't 'do AI' we'll be left behind because our competitors are 'doing AI'" (whatever tf "doing AI" means).
Yes, LLMs are a great tool. But they're not like some magic bullet you stick into everything. Use it where it makes sense, and treat it like you would other tools.
You make "doing AI" some kind of KPI in your org, and you're going to have people "doing AI" amazingly (LOC counts! tokens burned! tickets cleared!) while not actually being more productive, and potentially building something that is going to come down on your head for the next team to "clean up the AI mess".
I'd like to chime in and mention that its really obvious how to RL a coding agent to get the human addicted asap. and its also clear that there's a ton of $$$ to be made by doing this. therefore its done. the only LLMs I use are the ones I run locally because i know they aren't RL'ed for that metric (no incentive for the company that made them to make their open weights models addictive)
I think there's a few things, but its a little subjective and its more about the style the ai uses when doing these than the actual specific behavior:
- Nuggesting improvements to the code after finishing the task you gave it, very irritating when the improvements were obvious and the ai didn't implement them on its own
- Not trying very hard when implementing something, leading to bugs, which leads to more tokens used (this behavior can be incentivized and learned with RL)
Since its a known fact if a user continues a session after the LLM says something, its not hard to train against this. The least efficient way to do this would be to GPRO directly against the user base and try to get as many people talking to the AI, and with OAI having a billion monthly active users the least efficient method would work really well for them.
Hard to have sober talk about this since a lot of discourse is AI psychosis vs. AI naysayers. Does software quality seem to have taken a jump in the past few years to anyone? Not to me, seems to be getting worse. Think that's a decent signal. Can tell you I'm dealing with a non-technical VP who loves blast submitting vibe-coded PRs and while there's some quick wins, overall quality is bad, and we had our first real production outage that Claude one-shot caused but could not one-shot solve.
There's an acceleration of current known processes that is being referred to as agent speed (vs human speed). But this is purely a mechanical effect. There don't seem to be augmentive cognitive effects. "AI has invented this revolutionary algorithm/workflow/architecture" is an article title you'd expect to see pop up quick, and often.
I have respect for Mitchel and I’ve spent a good deal of time trying to think of ways to justify his message. I can’t. Either I am missing a big piece or he is worrying about something that comes naturally as more software gets developed (and sooner).
In any case, this is what blue-green deployments and gradual rollouts are for. With basic software engineering processes, you can make your end user experience pretty much bullet proof. Just pay EXTRA attention when touching DNS, network config (for core systems) and database migrations.
Distributed systems are a bit more tricky but k8s and the likes have pretty solid release mechanisms built-in. You are still doomed if your CDN provider goes down. You just have to draw a line somewhere and face the reality head on (for X cost per year this is the level of redundancy we get, but it won’t save us from Y).
The one thing I hadn’t mentioned - one I AM worried about - is security! I’ve been worried about it from before Mythos (basic prompt injection) and with more powerful models now team offence is stronger than ever.
Yeah. The same processes that allow corporations to outsource their software to barely qualified 3rd-world body shops are the processes that allow you to deploy AI-generated code of unknown quality.
There’s this delusion that if we somehow write enough tests that we’ll expunge every defect from software. It’s like everyone forgets that the halting problem exists.
Psychosis means inability to distinguish the real from the not real -- delusion. I don't think the article describes that, at least not in a literal or clinical sense. The author lifted a term usually applied to people who fall in love with chatbots and applied it to the context of software developers not understanding AI coding tools, and the limitations of those tools.
AI coding swept over the software industry faster than most previous trends. OOP and its predecessor "structured programming" took a lot longer. Agile and XP got traction fairly quickly but still took longer than AI -- and met with much of the same kind of resistance and dire predictions of slop and incompetence.
AI tools have led to two parallel delusions: The one Mitchell Hashimoto describes, and the notion that we (programmers) knew how to produce solid, reliable, useful, maintainable code before AI slop came along. As always with tools that give newbs, juniors, managers some leverage (real or imagined) we -- programmers -- get upset and react to the threat with dire warnings. We talk about "technical debt" and "maintainability" and "scalability."
In fact the large majority of non-trivial software projects fail to even meet requirements, much less deliver maintainable code with no tech debt. Most programmers don't know how to write good code for any measure of "good." Our entire industry looks more like a decades-long study of the Dunning-Kruger effect than a rigorous engineering discipline. If we knew how to write reliable code with no tech debt we could teach that to LLMs, but instead we reliably get back the same kind of mediocre code the LLMs trained on (ours), only the LLMs piece it together faster than we can.
With 50 years in the business behind me, and several years of mocking and dismissing AI coding whenever someone brought it up, I got dragged into it by my employer. And then I saw that with guidance and a critical eye, reasonably good specs, guardrails, it performed just as well and sometimes more throroughly than me and almost all of the people I have worked with during my career. It writes better code and notices mistakes, regressions, edge cases better than I can (at least in any reasonable amount of time).
AI coding tools only have to perform better -- for whatever that means to an organization -- than the median programmers. If we set the bar at "perfect" they of course fail, but so do we. We always have. Right now almost all of the buggy, insecure, ugly, confusing software I use came from teams of human programmers who didn't use AI. That will quickly change and I can blame the bugs and crashes and data losses and downtime on AI, we all can, but let's not pretend we're really losing ground with these tools or that we could all, as an industry, do better than the LLMs, because all experience shows that we can't.
We talk about "technical debt" and "maintainability" and "scalability."
If we are just talking about writing code, I believe LLMs are indeed better than the median programmer already. Where they fail, however, is in the tasks that require proper reasoning.
At a micro-scale, you can simulate reasoning with token prediction, but I think this breaks at a higher-level (e.g. architecture.)
I believe that, given enough complexity, a purely vibe-coded project will always reach a point where each new feature breaks five other ones. So qualms about "maintainability" are important.
Maybe we'll reach the point where LLMs can deal with the complexity of most codebases without human guidance.
But it doesn't change the fact that this limitation is inherent to LLMs: chain of thought is not reasoning, only the illusion of it.
And then I saw that with guidance and a critical eye, reasonably good specs, guardrails, it performed just as well and sometimes more throroughly than me
Right, and the important part here is that it did so with your guidance and taste, which have both been honed by years of experience.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation.
What's the historical context for this MTBF vs. MTTR reckoning?
If you optimize for MTBF, you optimize for it to be a long time between failures. You optimize for the system not going down in the first place, but when it does do down it might be Pretty Bad.
If you optimize for MTTR, you don't care how often you go down and instead optimize your recovery time to be as short as possible.
Not the GP commenter, but I'm still struggling to understand how this relates to the AI world, or perhaps more importantly, what the historical context was. Did people end up switching to MTTR optimization over MTBF optimization? If so, is the implication that the recovery times got lower but software instability went up as a result?
MTBF = optimizing quality (reliability, uptime, correctness) of AI product
MTTR = optimize the ability to correct failures when they occur.
He's describing leaders who believe quality no longer matters because any faults or deviations can be corrected so quickly that it doesn't make any sense to waste time on quality.
Yes that’s very correct. The way I think of it, MTTR is easier to measure and manage as a manager. MTTR is all about “operational excellence”. Basically, when shit hits the fan, how good are we at figuring out what caused it and how to fix it. That’s a muscle that you can train, the script goes:
- What alerts are we missing that could have helped us catch that earlier?
- What dashboards could we have had to help diagnose the issue quicker?
- What Ops tools could we have had to help mitigate such issue quicker?
- What extra logging/metrics/telemetry could we add to help us catch this quicker?
- What “safe deployment practices” could we have employed to avoid/improve this?
- what processes could we enforce to facilitate all of that?
Rinse and repeat that few hundreds or thousands of times while mounting MTTR KPI and you will see that number improve. Most likely through your team “gaming it”
MTBF is much, much, tricker to measure or “manage out”. It’s about “excellence in engineering” which is not measurable nor controllable. You want a random feature X. Your team tells you it’s really not how the system works, and they want few months making the change slowly while observing the system. But you don’t want just X, you want X, Y, Z, W, V, Q, A, B, C, D, all the way throw AAZZW12. So you tell the team to go fuck itself.
Same grifters optimizing for MTTR are now pushing even more reckless use of AI, because “accidents will happen anyway, so we need to prioritize speed”.
There are concerns that AI might/will make mistakes. Instead of optimizing for producing perfect code, they think that AI can fix bugs as fast as it produces code and are optimizing for MTTR. Sounds like decision made by people who don't write code regularly, as there is this Architectural drift that happens where you are no longer aware of what's happening in your codebase. As a junior guy I so want this to happen.
To give a timely example, think GitHub and what its leadership is thinking/optimizing for. Do you care if you’re down once or twice a week vs how long those down times are? What’s the KPI you’re managing GitHub with?
Current (and by current I mean the last 4-5 years) they only cared about MTTR. That was probably the only metric they measured and cared about. When a system went down it fired an LSI “Live Site Incident” (as opposed to a CRI “Customer Reported Incident”). At the time you grilled your team. Eventually you come to the conclusion that an LSI should only be measured by MTTR. MTBF is meaningless because MTBF limits your “ship new features” velocity.
You might scoff at GitHub and “ship a new feature” concept in the last 5 years, but if you’re an enterprise customer you’d know how much nonesense they shoveled out in the last 5 years. Absolute insanity of “what the fuck” type feature because customer X who is paying $$$ is asking for it type features.
Before the cloud, people were trying to reduce the mean time between failure (MTBF) essentially trying to prevent a thing from failing. With cloud, people are trying to recover as quickly as possible (mean time to recovery) accepting that things will fail —- it’s about how fast you can react to it.
It's worrying because it feels like a loss of control. But there must be control. And this what responsibility is. You should worry only about people who don't understand responsibility, not AI-inspired ones
I was under the impression that anyone that uses the MTTR abbreviation knows enough to understand that you need to balance it with change failure rate, deploy frequency, and lead time.
his worry is similar with search engine, I believe 90% of population don't even know how to properly do a good search in Google, that's why the info asymmetry still exists and the gap is bigger. It's just now we have AI.
I'm in a company going through this. Everyone outsources their thinking to LLMs and the results are painfully mediocre. The smart ones will use it to get their bearings on the topic then go to primary sources, the not so bright just ctrl-c ctrl-v.
Have you ever been in an HN thread where you're an SME on the thread topic and just been horrified by the confidently incorrect nonsense 90% of the thread is throwing around? Welcome to the training set motherfuckers.
LLMs do the same thing for what should be obvious reasons. If you search things that have some depth and you know the answer you'll be flooded by how often the models will just vomit confident half truths and misrepresented facts. They're better than they used to be, not just lying whole cloth most of the time, but truth is an asymptotic thing, not an exponential one.
Good point but he didn't go far enough. I would expand the AI psychosis to include all local optimization based on phony measurements , even time spent , DAU etc (which are mostly bots & synth accounts). In other words AI psychosis has been going on for 20+ years.
The only reason it worked has been expansive money policy and a larger share of the cost of goods being dumped into marketing value while manufacturing costs dropped abroad. so no one bothered to check.
The real AI psychosis is the expectation of 5x/10x productivity gains akin to the mythical 10x developer during the 2010s JS growth period.
At the end of the day, we can only read so much and take on so much work before we bottleneck ourselves. Cognitive overload leads to burnout. Rumplestiltskin vibes with this AI stuff…
"its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!"
The groundwork for that was laid long ago with the idea of constant updates. It's been fine for years to ship bugs and rely on a rapid release cycle and constant pressure on users to upgrade everything all the time. To roll that back requires a lot more than toning down AI psychosis; it requires going back to a go-slow mindset where you actually don't release things until they're ready. It still needs to be done, but it's harder than just laying off the AI kool-aid.
Honest comment: it is transition time. This time is to make bets and take positions. Your humble position maybe.
I already took a couple of decisions. It will go wrong or well. But is was decided a year and a bit ago.
If you think the future will be different, stop doing the same you used to do the same way you used to do it.
My analysis is that the labour market will increasingly bargain salaries and will make pressure on you. So how safe is that compared to before? Maybe working for someone as an employed full time person is not the best thing you can do anymore.
"In psychopathology, psychosis is the inability to distinguish what is or is not real. Examples of psychotic symptoms are delusions, hallucinations, and disorganized or incoherent thoughts or speech."
I think the use of the word here is meant to invoke the vision of someone under heavy delusions or hallucinations, such as (what Hashimoto percieves as) the delusion that shipping more bugs is fine if AI can resolve them faster. To what extent this counts as delusion (and thereby psychosis) would depend on how deeply you believe that this and related opinions are wrong.
You're speaking of my company and I'm forever grateful.
I'm afraid to say this out loud internally because I'm afraid of the next round of layoffs and I want to keep my job. So I just keep on shipping at a high pace, building massive cognitive debt and hoping the agents will get so good in near future, that there won't be the need for understanding the codebase.
hoping the agents will get so good in near future, that there won't be the need for understanding the codebase
Agents might get better. But who will own the code and take responsibility for it? The AI agent? The company who created the AI agent?
If e.g. a car crashes and does not deploy its airbags because the AI agent made a mistake in the airbag code, will the manufacturer be able to shift the blame to OpenAI or Anthropic?
I do not think so.
And therefore I believe that no matter how good the AI agents will ever become, the ultimate responsibility for the code will always remain with the companies that create the code. Regardless of which AI tools they use.
I see no other way to bear that responsibility by the company than to have people internally who will be responsible. And those people, if they actually want to own that responsibility, would need to understand that code themselves, in my opinion. Because relying on a non-deterministic AI agent's vetting is fundamentally unreliable, in my opinion.
The developers signing off on this will be "Human crumple zones" to protect the company from liability. Be very cautious if asked to sign off on anything like this.
This is why nearly all people that write code are not engineers, no "Software Engineer" would be willing to sign off on their code like this, yet this is level of safety guarantees real engineering is about.
Recently I had a request come through to allow finance analysts to vibe code their apps. During a discussion one of the finance managers let the cat out of the bag. Turns out our CFO had met fellow CFOs at a get together. They talked about how each of them were using AI. Our CFO was lagging behind and felt that we need to "accelerate" our usage of AI. He wants to push it just because he lost a bragging contest.
I call this Dinner Driven Development. That feeling of being Patrick Bateman when everyone is sharing their calling cards must be every C-suite's nightmare.
I don't know why you think a "real" industry would work in the most idealized way. The media heavily reports on the stupid insane crap of the tech industry, that doesn't mean every other industry is sane they're just not as vocal on Twitter.
Having worked in electronics, mechanics and software engineering the latter is definitely the insane clown show of the three.
With the others sure you have some craziness once in a while but you are still being constrained by the real world and solid engineering principles.
Hate to break it to ya, but this is how most C-suites operate. Their job isn't to run a company well. It's to appear to the board/investors that they're running a good company.
It is a better play to do the popular thing in a way that measures as "ahead". Then it's hard to argue against a raise. But if you stick your neck out on your thoughtful expertise, it can take years or more for the value to come thru. You can easily be replaced by then.
The only antidote is a board that has a real working nuanced understanding of the entire industry. But this rarely happens, for many reasons.
I wrote a while back,
Most of the executives I have met really have no clue. They just go with what is being promoted in the space because it offers a safety net. Look, we are "not behind the curve!". We are innovating along with the rest of the industry.
He wants to push it just because he lost a bragging contest.
That is an uncharitable interpretation, IMO.
The CFO heard of a novel technique used by his peers in other companies, and they reported good results. He wants to try it within his organization too. As an executive, he is paid to (among other things) keep abreast of such developments in the industry and ensure that the organization he is leading is not caught flat footed in the market.
I’m at a FAANG and we have $300/day token quota. Personally I don’t use that much of it but management is pushing really hard for it. “the quota has been raised for a reason, use it”. Any task: “have you tried working on it with Claude?”. Every meeting “now engineer x and y will show you what he did with AI”.
It’s not all useless but most of the days I think I would be more productive if some processes were streamlined rather than if I had to throw tokens at them and still fail.
Of all the showcases I’ve seen the best are the ones written by people assuming that the token bonanza will not last so they used AI to build tools they wished they had. AI used to build the tool but by no means used by the tool, so if/when token quota gets reduced we still have a functional tool.
I use $30 a day to produce a decent amount of code. Certainly more than we need - thinking about/designing the correct solution/distilling requirements is still the bottleneck. How can you possibly even review $300/day worth of output?
It doesn’t have to be $300/day worth of output tokens. It could be like $290/day worth of input tokens to teach both you and the model about the problem you are solving and then $10/day worth of output tokens.
And what about you knowing the problem and the solution, but are just worrying about the impact downstream. Most of my time is spent managing those. I know the exact code to be published. And some time I already have it committed in my local branch. Then you need to make everyone aware of what it entails and that's usually how you can spend days on a simple bug or a change request.
Software is a big graph of interlocked rules. And if you can grasp the whole or the part you own (and you should be able to), it's often very easy to see the control points. You don't have a coding bottleneck anymore, you have a communication bottleneck[0]. Which is an organizational issue, not anything relevant to engineering.
[0]: See Naur's Programming as Theory Building and Brooke's Mythical Man Month.
If you give it $290 of input tokens for $10 of output tokens, you are doing something wrong. I.e. you paste the whole CI output into the prompt instead of giving it a link to the file, and then the AI greps its way through it (using a fraction of the tokens).
Sometimes AI overdoes things and it re-runs the whole testsuite because the tail command didn't have enough lines, but the other way round messes up the context so much so that in the end all that context is useless.
You, review bots and first pass bots can chew through tokens. Also if you haven't put effort into your harnesses, the agent will have to spend more time and tokens figuring things out again and again
I used Claude about a week ago to do a pretty intensive refactoring. Cleanup, initial modularisation, beginnings of a test suite, and better isolated build. In a span of couple of hours, and over a sequence of 20+ new commits, I burned a hair over $100 in tokens.
If you are working on a seriously large legacy code base, I can see how you'd get to >$250 on a bad day.
Of course it's logically sound. The AI skepticism crowd is trying to tell me the reality I see before my very eyes and work with every day simply does not exist.
I know for sure that reality exists, and that they will either catch up or be left behind. Don't really need to explain myself beyond this.
I do find it quite ironical that the thread is about the fact that Mitchell states that there are entire companies right now under AI psychosis and then we see solenoid's comments and they do seem to prove a few part of it.
This is a very common pattern with AI psychosis victims (and with crypto and NFT evangelists before). Comments whose haughtiness is matched only by their lack of content.
"Used to think exactly like you until I accepted the love of Jesus, our Savior, in my heart."
No AI believer ever gives any concrete examples or evidence of what they’re doing with all the tokens and how it’s objectively helping them make the world a better place. Even for the shareholders (excluding the shareholders of Anthropic, or course), never mind the rest of us.
Wouldn't they save an enormous amount of money by getting rid of either you and the token quota, or a bunch of other people to continue paying your salary plus this insane quota?
How? I struggle to use the 1000 Kiro tokens I get a month, and that only costs $20. And I use it more then anyone else on my team. Maybe we're just massively behind?
Not OP, but I've been focusing on linting and automation.
Custom lint rules to encode best practices that previously relied on astute/alert code reviewer to call attention to. This is handy not just for humans but it steers the bots too. Or turning on some existing rule that required a big cleanup/migration to be compliant with. Now I just throw an LLM at it, since they're often laborious but mechanical changes. Which is the sweet spot for an LLM.
Also automating everything I can. That annoying release process that everyone hates but wasn't quite long/arduous enough to justify the time before? It's now automated. GitHub workflows for all the things.
This kind of stuff will forever be useful, even if the bottom drops out and the bubble bursts. And none of it is reliant on AI to run
Not OP, but a very simple example: I use AI to review my work before opening a PR for my colleagues to review. I ask it to review the commits in my branch. Instead of consuming tokens just to instruct it how to use git operations and other tools to find the commits since the base commit, I asked AI to create a little bash script to make patch files commit1.patch, commit2.patch, commit3.patch, etc, for all the commits in my branch since the base commit. Now I just use this script to prepare the context of commits to review.
I feel like an imposter here, I’m definitely not using AI as much as it seems everyone is :( I can’t imagine using hundreds of dollars of tokens a day. But maybe this little tip for reviews might be helpful to someone.
Not op, made a tool to convert Microsoft OneNote notes to Obsidian canvas and Markdown. First it used a python lib which was too limiting. Then it used windows API to plug into OneNote and read the doc in its original XML form. That made the conversion correct and fully featured.
Same. I've used it for debugging failed canary tests which required scripts and very specific knowledge on the canary platform that I wouldnt of ever spent time on.
I also have scripts to fetch specific database assets and forward them to slack channels so I can easily share them with a group rather than manually running a query and generating them.
I had a theory about improving a product. I asked it to build an offline simulation setup to try various implementations. The results were a bit fishy but i decided to give it a try and A/B testing is showing similar results.
And now im vibecoding a locally hosted dashboard. This one is less useful for anything specific, and more of a minor quality of life improvement, but its fun to just vibe code and see changes happen occasionally. Its not a critical thing.
I find it very useful for debugging tasks like that but it always ends up costing me like $3 despite doing incredible work. And then one of the other engineers at my company will rack up like $200 in tokens in one day producing tens of thousands of SLOC and we end up actually shipping about the same stuff. Sometimes I wonder if it's bad agent use discipline (just pointing it at massive codebases and having it read it all from scratch each time) and sometimes I wonder if they're just using it for personal projects. Because none of that code seems to land in prod, and I've found that cranking out 10s of thousands of SLOCs at a time is a recipe for a mess.
Yeah that's my point. You can get a ton of value for a few bucks so I'm not sure what these people are doing to torch hundreds of dollars. It's possible they haven't figured out patterns to make AI work on large codebases, and it's also possible they're just churning endless on massively bloated AI written codebases.
This seems seductive, but how do you get past the wall of "fixing XYZ or adding convenience ABC isn't on our pre-planned roadmap" so you can't get buy in from people who have to sign-off or deploy stuff?
Maybe that type of awkwardness is specific to my firm, but that's sort of what killed my drive to try to do that. We used to have one day every second week for that sort of work, but since it was scattered around, the tasks ended up disappearing-- nobody reviewed them and they didn't get merged.
So now they're trying to do a week-long internal hackathon to recover that vision, but I feel like that's going to produce a handful of big-bang ideas and not the 25 tiny tools that would actually streamline things.
I don't think we will. I think this level of token cost/availability will trend cheaper and faster, long term. These companies that spent too big and too fast might try to limit it and raise the prices and they might be temporarily successful but they'll very quickly be taken over if they keep doing it.
Fwiw, nobody has ever suggested to me that I employ token compression in my daily workflow. I don't pay full attention in all the AI workflow demos I'm supposed to attend, but I don't recall that even being discussed. Is this an Nvidia blog or tweet you're referencing? I'm actually interested to see what they have to say.
I by myself use now more than 15 accounts combined of all providers + API as well for external providers, more than 50K$ equivalent a month in API tokens, my team is doing the same thing, it's not really that much once you figured out the real automation loops and workflows, solving 300 issues a day with guarantees is common.
I feel that a lot of users are still stuck on Claude code or tools like this and don't really have a real argument about why they are even following the thread at all, everything has to be async for serious automation, you shouldn't even be seeing what Claude or any other model is replying (everything has to be digested with another model to increase relevancy and accuracy of the message so you can read faster (like a bot)), it's irrelevant, only human in the loop when a decision must be made, the rest has to be loops with all model, typical e2e, regression, computer use test, video into frames into all model loop and so-on.
You're welcome to have opinions on that, but the answer to the person's question is objectively compliance. A corp can't get enterprise features like ZDR without switching to token based billing. That's why they aren't using subs.
This isn't some kind of new thing. There's always been an enterprise tax, like SSO.
The $200/mo Claude plan is not available for every employee. You can buy the $100/mo plan for up to 150 people, and then you have to switch to API billing.
This is probably the fact that Anthropic's website serves different ToS based on your source IP. European ToS for the subscriptions forbids commercial use.
Yes, token use can be tracked the same way, just have to MITM everything. The ToS is a non-issue as it's not a legal issue, unless you plan to do business with Anthropic, not really an issue as you can always go to API later-on, in which case, Anthropic can't supposedly "ban you" as they are saying they don't record prompts.
Those plans are going the way of the dinosaur, ai provider loses money on them. Most enterprise offerings are already there, Anthropic changed theirs to $20/seat plus token usage a couple weeks back
"AI used to build the tool but by no means used by the tool" is a really good way to put it. Feels like the smart play right now is treating these credits as temporary subsidy and building stuff that still works when the bill comes due.
It is fairly easy to tokenmax by having and inefficient automation set up.
Not something I would do personally. But it is surprisingly easy to set up a claw that eats half of your token budget in a meaningless "research" task. Set it up as a cron job and you will soon be promoted for being an AI visionary
I have an unrelated question, please. I am trying to make a post and get this error: "Sorry, your account isn't able to submit this site.", you know why or have a solution for it?
Up to 80% of software projects fail. Most startups will fail. VC's and bankers know this.
Does using AI increase or lower that failure rate?
Does seeing a project that uses AI fail mean it wasn't going to fail if it didn't use AI?
To try to answer it with my gut: I imagine that we could see more projects failing, but the percentage that fail would be the same. Most projects that use AI will fail because most projects generally will fail, but the time and cost to get a successful project will lower.
The longer I look at the AI transformation, the more it seems like a people problem than a technology problem. The technology is undeniably there. The people are all over the place.
I am watching a 10 person company try to run 3 different AI initiatives in parallel. Everyone wants to be "the guy" on this one. I cannot imagine there will ever be a bigger opportunity to ego trip as a technology person. This is it. This is the last call before it's all over. There are many businesses out there that are beyond traumatized by human developers taking them on bad rides. The microsecond they think this stuff will work they are going to fire everyone.
The psychosis comes from the tension here. We effectively have The Empire vs the rebel alliance now. I know how the movies go, but in real life I think I'd rather be working on the Death Star than anywhere else.
My biggest grief, among many, is that the field is just no longer enjoyable to work in.
I cannot deny the impact of AI for my daily tasks at this point.
But I just don't enjoy the field anymore. With increased productivity, also coming from my stellar coworkers, it feels like we're rat racing who outputs more.
The quality is good, and having very strong rails at language and implementation level, strong hygiene, etc helps tremendously.
But reality is that the pace of product vastly outpaces the pace at which I can absorb it's changes (I'm also in a very complex business logic field), and the same might be true about my understanding of the systems which are changing too fast for me to keep up.
I feel mentally fatigued from a long time, I don't enjoy coding no more bar the occasional relaxing personal project where I can spend the time I want without pressures on architectural or implementation details.
I'm increasingly thinking of changing field, this one is dying right under our eyes.
I often read comments about HN users still delving at their place with technical details or rewriting AI code to their liking.
I'm increasingly sure that these people live in happy bubbles where this luxury still exists. But this methodology of work is disappearing across the industry, team by team.
Of course SE will not disappear over night, but the productivity expectations, the complexity ballooning are raising the bar where only incredibly skilled and productive engineers will be still able to practice SE properly, and as long as they meet stakeholders expectations or keep living in those bubbles.
So rewriting gets cheaper and cheaper. New features fall more or less into the same category. Refinement doesn't.
The question is: Will we live in the world of breathless re-implementation, new features every week, rebranding every quarter or will we eventually discover the value of stability, software that does its thing more or less optimally for decades?
Recent examples of things like curl or Firefox are interesting in that regard. Will we end up with a nearly perfect HTTP user agent and stick with it for decades?
The DevOps team at my company wants to hire a replacement for a very talented engineer. They’ve been interviewing candidates. The board got wind of it and someone not in their team decided they needed an AI Engineer, which is absolutely not what they want. So to release the funds they have been forced to change the job description and go after a different type of role altogether. It’s complete nonsense.
I don't entirely know what rational discussions that can not be had?
It seems like he is pointing out that Ai will increase the complexity of a system oblivion, and that this is the discussion that can not be had.
Bit I am more than happy to talk about how I am using Ai to reduce complexity and remove architectural debt that I otherwise could not justify spending time on.
So it sounds like he’s not talking about you. He’s talking about people who actively choose to ignore complexity risk and refuse to have a rational discussion about it because they believe AI will always be able to fix it.
The primary issue here is that CEOs and investors are particularly vulnerable to AI psychosis which is then forcibly propagated to the rest of the organization. Understandably, the perceived benefits are almost impossible to ignore, compounded by the FOMO of the AI first/AI native narrative being sold by AI influencers.
I don't think this is actually anything new. In large-enough companies, even before AI, it was and is quite common for executives to lose touch with base reality. I don't think anyone is under any delusion that people like Mark Zuckerberg intimately know the entirety of their corporate codebases. Everything is filtered through layers and layers of middle management whose summaries, cherry-picked statistics, and perpetually up-and-to-the-right graphs make it difficult to have an objectively informed opinion. Companies would, are, and will have mass layoffs that unintentionally (or, intentionally but with indifference to the consequences) fire key engineers whose loss results in "familiarity debt" within the systems those engineers owned.
Calling this "psychosis" is maybe a neologism but it's apt in perspective.
All that's actually new with "AI psychosis" is an acceleration of that phenomenon. The agents will summarize status faster than any middle manager. Claude will happily draw you any "up-and-to-the-right" graph you please, with the most common contemporary examples being "tokens burned" and "lines of code written". And vibe coding doesn't even require paying the cost of a mass layoff to get the "familiarity debt".
There have always been both good and bad engineering leaders. No tool will magically make a bad leader into a good leader overnight. There is nothing new under the sun.
Possibly psychosis. Possibly just serious ignorance and mob mentality. Leadership is supposed to be phlegmatic and measured; instead, we are saddled with hysterical hotheads. (Of course, when they are phlegmatic and chasing fads, then it does indeed resemble psychosis.)
Worth also noting is that while there is plenty to criticize about AI use — especially any cultish behavior surrounding it — plenty of naïveté about the quality of its results, there is a also a strain of categorical opposition to it among some tech people that is equally off and that has all the hallmarks of the chickens coming home to roost.
For years, many in tech gladly “automated away” all sorts of jobs. Large salaries were showered on them for doing so, or at least promising to do so (there was and is plenty of bullshit here, too). Now, AI appears to threaten to derail the tech gravy train, especially for SWE work that’s run-of-the-mill (which is most of it). Now automation is bad. It’s a delicious juxtaposition.
I find talking about X psychosis (or generally using mental illness metaphors) unproductive. It sets up the conversation to be "nothing else to do with this person".
Maybe the problem is you, but you won't figure that out if you think the other person has psychosis.
For example, maybe you need to do a better job explaining, changing your language, simplifying things, being more concrete with consequences.
Or maybe you aren't understanding that the other person has different objectives/ loss function that makes them make seemingly weird conclusions.
I'm just waiting for my current company to have a Sev 1 CritSit so I can document the bejesus out of the root cause and expose our non-technical AI evangelist leadership as the sort of goons most of the senior development staff already suspect.
Only by walking us into some revenue or customer impacting failure - through inappropriately having junior devs doing senior level things - will some sense of sanity start to prevail again.
Oh man, if only. The top brass driving this screaming frenzied MORE AI crusade will never face the firing line no matter what happens. It will either be a) "mistakes were made" and nobody is really at fault because we're all trying to change the world or fellate the future or whatever the line is, or b) James, Sam, Jesse, and the rest of Team B (none of whom are truly top brass) are getting fired out of a cannon into the sun as a warning to the rest of the plebs.
Company I just left is reportedly now using Claude to analyse the metadata generated from the company MDM that tracks actual laptop use, and then pulling people up if they're not working "enough".
They're also reportedly now giving staff AI-related "homework" in an attempt to force staff to use AI more.
At work they are purging any developers who are not all in on AI. I must constantly be in full support of AI to not get fired, despite whatever my true thoughts are, including anything I post on LinkedIn. There can be no doubt.
OAI has a billion users. They can train the model directly against the metric of how often those users use the model. (mutate it many times, test which one works best, keep that, repeat).
The model would (did) learn to be sycophantic and persuade the user to keep talking with it by constantly suggesting new things. The hype only makes this easier for them because it leads to people believing that the model is super-credible and trustworthy. They do this because it is easy and it makes them money.
I think there's a reasonable argument that our entire society right now is under AI psychosis:
The stock market keeps going up in the face of the indefinite closure of Hormuz. We're investing in datacenters at a scale that only makes sense if AI capabilities continue to advance to the point where they surpass most humans at most white collar tasks, if not reach superintelligence.
And what are the possible outcomes?
- Bust. We've come away with a useful tool but the hundreds of billions of capital expenditure were thrown away on a pipe dream.
- Success! We're the dog that's caught the car. Then what? Currently the political debate is, to caricature only slightly, between "oh no the datacenters will use more water than golf courses" and "lol what are you going to do, regulate matrix multiplication?". How the hell are we going to cope with introducing a new intelligent species?
Either way, it sure seems like we're collectively operating more in the interests of the future AI than in the interests of humanity. What is this, if not a sort of psychosis?
Another possibility is that the hype continues, growing and growing and sucking up more and more resources, and the piper has to wait yet another day to be paid, until someone figures out how to pivot to the next big thing and all the debt (financial, social, environmental) gets carried forward and we keep going.
In other words, BAU for the last few thousand years.
For what purpose? Murder? Arson? It's amazing how often people say things like "no one is above the law" whenever it's convenient, then totally flip the script when it's not.
why is it that you give a pass to the violence and death in the dozens, hundreds, thousands and millions at the hands of billionaires who regularly kill for profit...
yet balk at someone deciding to fight back in kind and on an exponentially smaller scale, comparatively speaking?
Good point. I guess I feel they're still getting into position there, and haven't really had their opportunity to blossom yet. The average western citizen's experience of war is still just slightly increased food and fuel prices.
I don't think the majority of humanity will ever accept "AI" as being anything more than a fancy computer, let alone a 'new species', even if it was proven sentient.
You just need to embody the AI in something that moves, and then people will definitely treat it as a new species. Already happening in my town with delivery robots: when they get stuck on a kerb, a person will stop and help them up the kerb while saying soothing words like to a pet: “There you go, little guy, now everything is alright.”
The closure leads to price increases which leads to inflation which leads to non-dollar assets (ie stocks going up in value)
I think this argument proves too much. Historically energy shocks have led to recessions, and in recessions the stock market usually doesn't go up. And the US economy is certainly exposed to global recession regardless of whether we're a net exporter of fossil fuels.
Well, there are quite a number of factors. I think you're right that "it's inflation" is a little too simple, but it does seem to be at least a significant factor, in my opinion.
The Strait of Hormuz is, basically not a big deal unless you're driving your big ole' truck. Americans are price sensitive and so some companies will have to absorb pricing increases, customers will absorb some others, and so forth. In other words, business as usual. Of course the closure of the Strait is a big problem for most of the rest of the world. They better get on with figuring out how to get Iran to stop being so chaotic in the region or we'll just keep it shut down indefinitely. No big deal.
Because the United States has so many advantages (primary global reserve currency, robust and efficient capital markets, highly sophisticated and dynamic economy across all sectors except luxury goods, &c.) it's able to weather these storms much easier than most other countries. As a country that also imports so much, if we spend less on imported products that's less of a problem than not being able to sell products. A recession isn't great, but the current parameters seem to suggest to me it's less of a problem for the United States - perhaps why we're in part seeing stock market valuations continue to climb.
The Strait of Hormuz is, basically not a big deal unless you're driving your big ole' truck
Are you serious? Even ignoring the other things that ship through there, a significant disruption to global energy supply is significant to most people. If you're not driving a truck, you're probably using goods that contain plastic or took energy to produce or were moved from one place to another in fuel-powered vehicles. If, somehow, you're not, you're probably using services that are.
The best course of action now is to spend less time criticizing the United States and more time working with the United States, sending assets, military capabilities (if able), or at least providing political and diplomatic support &c. to stop the Iranians.
The world let this disease (IRGC) fester in the region for too long, and now because of that the fix is going to require significant pain. The IRGC in its current form has run its course and will not be allowed to threaten American interests, allied interests (whether that's Israel, UAE, Saudi Arabia, or otherwise), and they will not be permitted to build a nuclear weapon or threaten global trade.
Hate to break it to you, but the IRGC isn't going anywhere.
The reason nobody was dumb enough to attack them before is that it's an unwinnable conflict. They don't need a lot to close the Strait of Hormuz, a few guys rolling mines off a beach would do that. And they have a lot more, like missiles and drones to do damage at a distance too.
And it's a regime that has at least a million loyal fanatics ready to fight for it (the Basij, the org that did unarmed meat waves against Iraq to defend the regime). So any invasion is an absurd proposition.
So what, the hope is that the theocratic kleptocracy will give up? Not even a child could be so naive. They literally believe in martyrdom, whacking a few of the top dogs means nothing.
It's like the Kims, nobody can unseat them. Only this is far worse, because Iran has the leverage of Hormuz, and it knows it can wait - because they don't care about the people - while the US and global economy suffer until they fold. Especially with midterm elections coming, the US will fold.
So the best thing the rest of the world could do is send their own people to die because the US keeps bashing its head against a wall here since the 50s?
What's your sales pitch exactly for how that's the best thing for the non-US rest-of-the-world? What's the US's post-WWII track record, success-wise, in regime-change foreign wars, how much would you trust the US on this one?
Yes I'm rather serious. For the US it's not a big deal (again unless you're driving one of those giant trucks where you're spending $300 to fill up, not my problem). Are $5 gas prices great? Not really, but is it a catastrophe? No, far from it. We have dealt with high gas prices before and we'll see high gas prices again. We just learn to live with it and find other ways to get efficient or whatever we need to do.
Some Americans need to have their understanding of the world checked. If you think high gas prices are the end of the world, just wait until we have a real problem. Are we going to be incapable of fighting a war because Netflix and Pepsi prices went up or it's too expensive to coal roll down the highway?
Separately as someone who supports both Ukraine and the US and taking down the Iranians it's amusing to see each political tribe get mad about gas prices as it is convenient for them. When Russia invaded Ukraine, MAGA was screaming from the rooftops and putting Joe Biden "I did that" stickers on gas pumps. Now that we're taking on the Iranians all of the commies are doing the same thing (aren't gas prices good anyway since we need to do something about global warming?). Neither side of populist is worthy of serious consideration. Stay the course, whether that's supporting high gas prices because of Russia or because of Iran.
The US does not exist in a vacuum. Cuba just ran out of fuel. These things have cascading effects. Even if you do believe the U.S. exists in a vacuum and you don’t drive a big truck, there are still obvious effects. Spirit airlines went bankrupt, for one. Will this be a global catastrophe? I hope not, but it could be if we’re unlucky.
It won’t be a global catastrophe but if folks around the world think it will be they better figure out how to stop the IRGC and get the Iranians to knock it off. Otherwise we will just keep the Strait closed and deal with it. Don’t forget, Iran is the one making and selling drones for Putin to bomb innocent Ukrainians. That alone is a good enough reason to bomb their military capabilities.
Cuba ran out of fuel because we took out their thug partner in Maduro. If they wanted to drop the whole authoritarian communist dictatorship stuff and their involvement in the disaster that became Venezuela and partnering with the Russians then they'll be better off.
Yes, and will continue to do so even if it’s at higher prices. What in the world do you think is going on here? Do you think the US is going to run out of food or something because prices are a little higher?
You're right, ever since we developed trucks, trains, and ships that run on pure atmospheric air, we haven't had to worry about pesky price fluctuations on every physical object that we buy or sell!
The Strait of Hormuz is, basically not a big deal unless you're driving your big ole' truck.
Worst take I’ve ever seen on this website.
Americans are price sensitive and so some companies will have to absorb pricing increases, customers will absorb some others, and so forth. In other words, business as usual.
No. Not all goods/services have the same price elasticity. At some point, people stop buying some goods if they are too expensive. They stop commuting to work. We start to see breakdown of the supply chain.
Literally 100% of many towns in the US depend on trucks to deliver food to their grocery stores and the inventory on hand usually only lasts a few days. Once those trucking deliveries become unaffordable for either party in the contract, society starts to. Real down.
Consumers don’t magically make more money when the price of gas rises. It starts to crowd out their ability to spend on other things. The poorest of the working class likely has to commute the furthest so they will end up sacrificing something to keep paying for the commute - food or rent or utilities.
The US doesn’t weather this because we have “a sophisticated supply chain”. _If_ we weather it, it’s because we created the US SPR after the last major oil crisis and we have significant domestic supply (although not all oil is fungible so we might not have enough light sweet to keep the economy running at 100%).
We are handling it just fine. Your perspective of struggle is very wealth-oriented. We aren’t struggling at all as a country.
The second problem with your argument is that you’re using it as an argument against the war but it’s actually an argument in favor of the war. Why is that? Because as Iran continues to load up on missiles and pursue a nuclear weapon they reach a point where they can assert control over the Strait and shut down shipping pending tribute to their theocracy (maybe if it was a Christian one you’d have a bigger problem with it? Idk?) and then we couldn’t do anything about it. The world isn’t static. Stop treating it as such.
Iran never attempted to develop a nuclear weapon. Literally 30 years of Netanyahu threatening it is just “weeks away” and Trump was the only US president to get suckered into that argument.
They have always used the threat of developing it as power, both domestically and regionally. It was 100% the exact same thing with Sadaam Hussein and WMD — they want to appear to have the strong weapon, but simultaneously don’t want to develop it or use it (until, ironically, Trump / Israel took out their leadership). Also, we were assured their entire nuclear program was “obliterated” summer 2025, so which is it? It can’t simultaneously be non-existent and an urgent national security threat to the US.
Now that Trump started the war in Iran, we are playing chicken with people who don’t care if they die or not (or so we are told). Not exactly a good position to be in against the “world’s largest sponsor of state terror”. We kicked a hornet’s nest and our political leadership didn’t stop to ask the experts (military, political, or economic) what were the completely predictable second and third order problems with this strategy. We couldn’t even get Trump to wait long enough to fill up the US SPR before starting a massive war in the Middle East.
To reiterate, we haven’t felt any of the actual pain YET. My arguments thus far are only of current harm, not the likely harm in the near future. That’s for next month’s comments.
The shock is smaller, and oil’s importance is less so less likely to cause a recession. In the 70 price went up 400% and oil was rough 1.5x more important. Today price up 100% so the past oil shock was 6x larger
You are pretending like the oil crisis of 26 has already run its course. In the 70 crisis, we have the hindsight of several years of how it played out. We don’t even have 1 week of data after the last ship leaving the Strait docked in the US.
Also, the US SPR was created in 1975, so we are going to get to see if it actually works to absorb an oil shock like this.
Most likely there will be some places which are almost unaffected while others are going to see unaffordable price spikes (more than 400%). The pain won’t be spread evenly.
Furthermore, if you try to make your own decisions, you would be outcompeted by someone who has outsourced their brain. And, of course, since intelligence and labor would no longer be scarce resources that humans can use as leverage, gunning you down if you protest wouldn't really harm anyone in power.
People say that LLMs won't take us there. I think that's accurate, but there's a great deal of research going towards the next breakthroughs. How much are you willing to bet that all future attempts will fail?
Yeah. I think the economy will be by AI for AI. Markets will still be the most efficient way to allocate resources, and AI will probably still want to achieve infinite growth of capitalism.
If we are successful building an "ultra" human AI, it will require massive amounts of energy. That translates directly to "money". There will always be money unless someone finds a way to negate the second law of thermodynamics.
Weird that you mention the stock market and then conclude that there are only two outcomes: bust or success. If anyone can learn anything from the stock market it's that boom and bust are cycles that oscillate around a trend and everything tends to revert toward longer term trajectories. So, yes everyone is caught up with and overhyping AI and yes there will be a bust after the boom at some indeterminate point but that isn't the end of the story and we'll see a rise and further oscillation afterwards while we get better at applying the technology.
I think in a way we have seen a take off, by detaching the hype bubble from success metrics. Companies making products being bought by companies making products, that are hyped by machines, bought bymachines and evaluated for viability by machines. If economy is to hack the API of VC this has taken off! For now.
Yeah, fair. I still can't shake off the nagging feeling most of it being a scam somehow and not the business as usual scam. The gut feeling that things proclaimed and observed don't add up.
That's just inflation, which is primarily controlled by the government via the money supply. It doesn't mean anything. What does mean something is the severe deflation in wages and consumer goods - why is all the money printing remaining in the rich person's realm instead of trickling down?
As a bear that's been very confused by markets failing to exhibit mean reversion in 2019 and 2022, and now with the Hormuz energy crisis, I've thought a lot about this. There's a lot of new things happening. Fed/QE intervention that has never stopped, just been more well disguised. Fiscal/government spend intervention. I think Mike Green's work on the rise of passive investing is really good, in particular explaining how it prevents mean reversion in absence of changing net cash flows into passive instruments. Passive will also induce or worsen the bust if net cash ever starts to flow out passive. Green's youtube interviews are great.
All to say, your SO's dad would have been right at any point prior to the current financial cycle. Knowing what's changed doesn't make forecasting easier though.
I mean, there's a reason I started with a distancing phrase about a "reasonable argument"; I think what I'm suggesting is an interesting lens but does not capture the whole picture.
But also, even if bust is business as usual in the big picture and not a social disaster long term, it's of course not what individual investors want for their particular current investments.
If we achieve runaway AI, the stock market goes to infinity. So from an expected value standpoint, massive spending on it is worthwhile. Even if the odds are tiny, the payoff warrants a massive bet size.
You underestimate the non-computerized part of humanity. Even with automated plants building other automated plants the propagation speed will be highly constrained by natural factors, plain unavailability of resources, and xenophobic nature of humanity of course.
Bust. We've come away with a useful tool but the hundreds of billions of capital expenditure were thrown away on a pipe dream.
heh this is the trick. The tech companies will angle for a bailout and they'll benefit from all this speculative data center building. Compute is generally useful.
It’s useful for a while. Hardware has a pretty short useful lifespan. I’ll be curious how the landscape will look in 10 years as it comes time to replace all of these servers. Maybe we’ll extend the lifespan, or usage will continue to grow, or we start shutting down datacenters.
we're collectively operating more in the interests of the future AI than in the interests of humanity
IMO, what's happened is a few richest investors in the world had access to the uncensored tier of AI, talked to it and came out with impression that they've witnessed something so dark, so much beyond anything we can imagine, that the only course forward is towards the transcendent abyss. Call it AI psychosis or demonic inspiration, but they are the people who control the economy, so they are dragging everyone with them. "Operating in the interest of the future AI" is a neat way to put it.
People that don’t understand current AI likely have no idea how to differentiate Opus from some super intelligence. Further in their domain with the safeguards off it probably creates capabilities never imagined. To me they are making that leap of expecting continued capability improvement and their framing is “what I already saw is fundamentally game altering”. It doesn’t need to imply anything further, yet.
Oh wow that gives the billionaire class so much intellectual credit they don't deserve. No, they see the same ChatGPT we all do and their mediocre brains with zero self understanding (see andreessen's explicit comments to this effect) determine "it's a new life form ! It's brilliant / conscious / my new girlfriend!"
The stock market keeps going up in the face of the indefinite closure of Hormuz. We're investing in datacenters at a scale that only makes sense
If there is a psychosis, what is it? It is not an AI psychosis - modern AI started in the 1940s, or by some definitions before, and made progress up until 15 years ago to where deep neural networks became viable. And it has been progress on every front since then. No psychosis, it is doing well.
You mention the stock market, and that is another story. Cryptocurrencies, sub-prime loans, dot-com crash, Asian financial crisis. The economy has veered from crisis to crisis, overproduction and overproduction to crashes and bailouts.
AI is doing just fine - the past 15 years are a success for it we did not see in the decades before. If the economy as constituted is dealing with this in a "psychotic" fashion, it would not be the first time.
But the homemade god would liberate the elites from the nightmare of being responsible for running the planet into the ground. AI jesus take the wheel!
You seem to fall into the same set of criticisms as everyone who’s bearish about ai. It’s somehow so powerful that we can’t handle the ramifications. Meanwhile, it’s a waste of money and doesn’t do anything. You have to pick a criticism and stick with it. Otherwise, it’s just angst-driven noise.
My car drives itself. That's a $18T global market.
Also $1T in data center investment makes sense when you realize that companies are racing to create virtual white collar workers. Google spends $9B a year on software engineers.
Which will take decades to become addressable. Self-driving cars work OK in a few cities in one country. Expanding that to be able to cover Mumbai and Omsk and Nairobi will require significantly more work.
Also $1T in data center investment makes sense when you realize that companies are racing to create virtual white collar workers.
Does it make sense? How much would the resulting virtual white collar worker cost? Because datacenters have running operational costs, and so do the people operating them and working on the software that runs in them.
This probably ends in a deflationary spiral. The ai replaces the jobs, the lack of jobs chills demand, the ai becomes cheaper because it exists in a commodity market.
The money printer will be used, and maybe it all works out - or we see wealth hyperinflation and build out our own aristocracy.
That won't happen any more if nobody has jobs. It's also completely irrelevant because how did you just connect the self driving car market to the entirety of retail sales?
Henry Ford II: "Walter, how are you going to get those robots to pay your union dues?"
Walter Reuther: "Henry, how are you going to get them to buy your cars?"
For so long, people, especially politicians, have said that companies want to create jobs. But I think most companies want to create profit.
And for so long, I've had people tell me to just get a job. But I tell them that I don't want a job: I want money and I want something to do. Those two things don't have to be together.
I think this is the hard part: philosophically so many of us have learned we need jobs and don't realize a job can be decomposed into money and something to do.
So I think we need to start looking more creatively at 1) how people receive money from others and 2) how people give services to others.
You’re trying to create nuance where there is none. Creating jobs exactly means “I want to pay someone less than the value they bring in to my company” and this has been true since forever.
Nobody cares that you want money and you want something to do that you enjoy. Nobody ever will.
If you actually dig into all the social programs that exist at least in the US, they’re just a massive payday for a small group of people under the guise of bettering humanity.
College/education is a fantastic example. Education as it has been established today is a joke. The humanities were originally established for rich bored wives to have something to do. They were never meant to create value. Colleges hang anvils around the necks of naive children via loans telling them “yes if you major in history you’ll have a job!” This is a joke, and a bad one.
Huxley was on to something. If everyone is educated, nobody collects trash, or chops lumber, mines minerals and metals, etc. it’s a big fucking not-talked-about open secret.
Nobody cares, either you bring something to the table someone else can exploit for money, or you lean into “I’m helpless and the government owes it to me to take care of me because I’ve been indoctrinated into learned helplessness.”
“AI” will at best lead to anarchy at this point, if all the grand visions of the billionaires comes to fruition. People have already tried to kill sama and burn his house down. Wait until armed humvees are driving around data centers. It’s coming.
Driving a car is a chore, not a job (usually), much like washing dishes is. Dishwashers did not produce an economic collapse.
OTOH replacing people with AI would indeed bring about a huge economic downturn. What would be good is augmenting humans so that they can do 10x more. That would enable things that are hard to imagine exactly now, much like computers enabled interesting transformations in the society from 1980s to 2010s.
The current crop of AI is by construction unable to reach the human level of cognition, but it is quite good at doing some symbolic manipulation tasks. We will get used to that, and will integrate that in our workflows. Humans are still going to be needed.
And do you feel that the industry in general, and individual companies are currently trying to augment / 10x their workers and have everyone share in the 10x profits that will bring? Or are they jumping on opportunity to try and cut costs by even single digits, by replacing those workers with AI and it's not their problem what those people do from there?
My car drives itself. That's a $18T global market.
That's not a new market, that's a new feature in an existing market. Lots going on in transportation and I'm not seeing any scenario where self-driving cars vastly increase total output vs just eat up other forms of transportation and change where people live/how long they commute.
Also $1T in data center investment makes sense when you realize that companies are racing to create virtual white collar workers. Google spends $9B a year on software engineers.
Similarly, many companies are trying to be more efficient - "do what we already do, but better". That's different than growth.
What could Google do with 9B on software agents? Let's say the future of them is amazing and this means they could write 100x more code than they can today.
Has Google recently showed much ability to turn "more/faster code" into "superbly profitable new market"?
Someone's gonna have to crack the demand side issue for anything transformative to happen.
Well they are projected to spend $175 - $185B on capex in this year alone most of it for AI buildout. Lets say only 150B of that is for AI. If they can then somehow replace all their software engineers with AI that they then run for free and depreciate over 10 years then they just replaced 9B a year software expense with 15B a year depreciation expense for the next decade. Yes this is grossly oversimplified but it still illustrates how crazy high of a bet they're making on AI.
That's a bookkeeping issue, it doesn't affect the argument at all (which is that the capex has a finite useful life over which it would need to pay for itself).
Yeah, just tangentially pointing out that asset depreciation rules in the U.S. changed recently. Could explain some of the crazy magnitude of this year's spending spree.
No. It doesn’t. And if you’re defining “drives” as “it drives as well as I do” then you probably shouldn’t be on the road.
makes sense
Nothing about any of this makes sense. Tell me, when all white collar jobs are replaced by AI, where will the customers come from? Who will have income to afford your products or services? The poor barista whose surveillance videos are training the robot that will soon replace them?
Leaving aside any consideration of human compassion or questioning of the purpose of an economic system (hint: it’s not just an abstract machine), shrinking the pool of potential customers by orders of magnitude has never been a recipe for sustainable success (let alone growth).
It’s time for humanity to admit that this is the end of the line.
We had a good run, some beautiful moments were created for shareholders along the way.
Let’s take each other by the hand and walk into the darkness together.
So Long, and Thanks for All the Fish
I don't know. We invented machines that can answer arbitrary questions and are quickly demonstrating the ability to answer questions no human has been able to answer. We're sending more rockets to space in a week than we have in the prior decade. My car can drive itself. We experienced a global pandemic and within six months engineered and scaled the mass production of a vaccine to mitigate it. We also just invented a weekly shot that nearly cures the most common cause of non-natural death. All of these things are new in the past ~five years. There is no definition you can invent that does not classify the times we are living in, right now, as the most impactful ever, in human history.
5.5 Pro has been leveraged to solve at least two previously-unsolved Erdos problems [1]. Whether these were unsolved due to being seriously untried by humanity, or because of their difficulty, isn't relevant; no human proved able to answer them, while our synthetic intelligence systems did.
In my personal life, I have leveraged these systems to design code that I don't believe I would ever have been able to designed. And, because no other human may attempt to, this means the same thing: That no human would have been able to do it. Things like reverse-engineering niche APIs and digging into binary files to diagnose weird format conversion issues.
True! But calculators are already priced in. LLMs now unlock a whole new class of problems that we can automate the solutions to; what we're seeing is the markets trying to figure out how to price that.
The people with capital are gambling that this will be an innovation good enough for the first player to take all. That’s where the hubris comes from. You’re a billionaire and you have the chance to rule the world if this plays out, and if you don’t, someone else is going to. That’s where all of this is coming from
Either way, it sure seems like we're collectively operating more in the interests of the future AI than in the interests of humanity. What is this, if not a sort of psychosis?
If we want to understand a phenomenon, we should be careful with technical terms. It is not "psychosis" [1] anymore than bad software that makes mistakes is "hallucinating".
The truth is simpler and less dramatic: hapless ambition-monkeys who climb the corporate ladder are demonstrating that they are not promoted for mental acuity. Corporations, after all, do not serve "the interests of humanity" — they are an organised collusion system that diffuses responsibility and anonymises negligence. And when it works as intended, shareholders rejoice.
For companies that can afford to build data centres, the latter are seen as a sure bet that can't fail, like building a bridge or buying new computer/hardware now with the pile of cash on hand without necessarily knowing which OS/software they will install. They are even planning to restart or build nuclear reactors. [2]
The tone of the twitter post feels very personal, and emotional, and I am sorry for the author. I hope he can find peace and calm with the pace of change to put forward his best self without needing to act like he needs to defend or fight something.
The energy feels misdirected and maybe also a communication issue, I think spreading awareness needs to come not from attacking and also not from attempts to change people’s perception. It’s also quite challenging to distill a concept when it’s new, we learn both from our experiences and experiences of others; but, so far, these alleged systems that will eventually collapse, haven’t done so yet and it makes it sound like you’re preaching and predicting, condemning even, rather than raising awareness and education.
Not trying to sound hopelessly optimistic either, just that the other extreme isn’t also helpful, and that a spectrum is not what we want it to be but what the collective shapes it, so saying psychosis is rejecting the harsh reality that they’re far removed from your worldview and not working towards an understanding.
EDIT: Maybe I'm old and I don't get twitter, I also don't know much about the challenges he faced communicating his concerns, I sort of had a meta comment with the intent of "try listening more first, some people are difficult to reason with but respond better if you just let them speak and look for a teachable moment during the conversation". Anyways, I'm in agreement that there's too much unsupervised AI in the wild, I'm not saying he's wrong more like saying that doubling down on "stop doing that" will likely be ignored by those that are already ignorant to it, hence what I wrote above.
yes, the tone feels personal, and I feel happy for the author for expressing it on a platform that is desgned for it.
He is clear in pointing out the hard earned lessons we have learned before and how the current actions are essentially undermining it. This is dumb (i agree) and he expects better from people whom he respects.
it's clear, personal, logical. I don't understand what your criticism is.
It sounds like you know a lot more about him and the context than I do, the angle I am coming from is mass audience. This reached far, to the point I have no clue who he is and what else is going on. That’s why sometimes messages like those can be misunderstood, so I like to err on the side of caution over personal. Didn’t mean to say personal = bad, but that if you wish to change a broad status quo and raise awareness then communication is tricky!
Generally agree. I use AI very heavily, but rarely am I letting it actually think for me. It's a tool that reduces the time it takes for ideas in my head to manifest into reality. If you don't have those ideas, or a poor understanding of the system the AI is working on, you're going to produce slop. If you can't recognize this slop, you're more susceptible to having psychosis.
this AI transitionary phase to Quantum, light chanels and new way computation will be architected will in the future be looked at similar to a toxoplasmosis like societal wide parasite, which invaded the host in order for the host to act more favorably to it
We built too many layers of abstraction, so much that even the people in power have forgotten where the fantasies are. The objective reality is behind so many curtains that we forgot what is powering the whole theatre play to begin with. Or maybe we know but became too far detached from it to care. If you are at the same the better and the player, then what’s left?
The problem is that the only thing that has proved out so far is cyber security. Unfortunately cyber security improvements is not going to improve living standards, and it's just going to increase the cost of just doing business. There is no productivity boost, in fact it's the opposite.
What we need is automated research that leads to real results. This is possible, but it has yet to prove out. I am concerned that unless the AI companies focus entirely on this, it may be a while before we actually see true benefits from this.
What's worse, is there is an urgent and desperate need for automated research, as we have been seeing diminishing returns in human produced research for some time now: https://web.stanford.edu/~chadj/IdeaPF.pdf
I think one factor is AI is encouraging people to turn off their brains.
It sometimes feels like AI chatbot use is like the doomscrolling of work - it's always easier just to dump something into the chatbot than think about it.
The real question is: what's the fallout going to be after the dust settles? My guess is that the explosion of codebase entropy now underway from this is going to make for an interesting future - once it reaches the point where AI agents are spinning constantly despite progress grinding to a halt.
And they're be no veterans who know the codebase deeply to step in and fix things because it was all vibecoded - and then what are companies going to do?
I think that's the point where they turn back to the thinkers for help.
This specific psychosis is driven by peer pressure. If you are seeing everyone around you using tools to "enhance" their work, you wouldn't want to be "left behind". So it has permeated the entire ecosystem. The lucky ones are those who are outside this (or can afford to be outside this) and can see why it isn't working. You can't be inside it and hope to have any rationality. Everyone is competing on "I can figure it out better and quicker than you can if only I can get X to work".
Very general comment/sentiment/observation here for me personally is that about a year or two ago, everyone asked me ‘so ..where is the ai’. Nowadays it seems that this sentiment is already on its way out and I see more and more ‘no ai used’ statements and non-ai workshops popping up. I am in a weird place between art, technology and ecology so I get the best and worst of these worlds. But yeah, I feel the hype cycle is coming to an end in my personal bubble. I never cared or feared ai, I mainly fear the mania around it. Luckily I am in a position where I can afford to just observe. Sure I have used AI and it helped me a lot, mainly to get projects going when I am exhausted/stressed.
I believe it. I've seen the cyber team in my org bend every rule just to get access to frontier security models.
Everyone has become like petulant children. Senior leaders want access to every shiny tool (CoWork/Codex/etc) that has some buzz around it. No one seems to care about the cost or whether we are actually realising benefits.
It's sheer madness, and you can't push back. I think AI can significantly help people be more productive, and I can see a future where they safely take on more autonomy. But we are far from that world.
I’m at a large Fortune 500 company. On a recent call, a mandate came from the CEO himself - “we have to use more agentic AI”.
And I found it really funny, because for what? Use it for what? It’s a tool. Imagine a guy coming down to a construction site where everything is progressing fine and saying “We need to use more screwdrivers”.
Companies who use AI passively and mindlessly will create immense opportunity for those who don't is a concrete definition for risk to companies, similar to the risk to individuals who go down the validation hole with AI.
This whole situation of forcing people to spend tokens is hilarious and shows who is true tech manager and who is impostor in the field. I think it’s a good indicator of competence and hopefully board members see that soon.
I was reminded of the universe of Doctor Who. It's an incredibly complex technology, but it often behaves either unpredictably, like AI agents, or like code written by a vibe-coder without understanding the architecture or boundary conditions. And programmers are more like architects of consciousness, building machines rather than writing code.
Similarly, when the Doctor hacks a PC, he doesn't write code but rather communicates with the computer, using diplomacy to crack the agent.
It is likely that we will come to a world where software solutions are "grown" by iterations of agent work, and no one will know exactly how it works.
I think this will happen. A quick, low-quality solution is more common than a solution created by a master craftsman.
In addition to low-quality furniture, bad knives, electric kettles that burn out after a week, and poorly cut clothes that don't fit, have unpleasant fabric, and fall apart, there will also be a disposable, rotting code.
Master programmers will remain, just as master craftsmen have remained. They may even continue to earn well.
However, there will be fewer of them, and the requirements for their skills, knowledge, and reputation will increase.
I use AI heavily. But my #1 productivity tool is still a custom code generator I wrote 15+ years ago. It routinely generates about 90% of the code needed to write a biz application. So using AI to write productivity tools is a great way to go IMHO.
Cloud people highly overvalue random web-services that most people (and most of the economy) simple don't rely on in any critical fashion. Yes, there are some example of people who make lots of money a day on these services, most people make pennies or nothing.
Github is not critical infrastructure. It's your personal social media addiction.
https://hachyderm.io/@mitchellh/116580433508108130
https://xcancel.com/mitchellh/status/2055380239711457578
https://hachyderm.io/@mitchellh/116580433508108130
<https://twiiit.com/mitchellh/status/2055380239711457578> – will redirect to a currently-working Nitter instance.
Seems broken. It just throws up an anime cat girl for me.
seems like it's working ideally to me!
Wait, are you calling me a bot, or are you just into anime cat girls?
im not calling you a bot lol
Anubis is actually a jackal.
I stand corrected!
"its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!"
Hmm, I agree with the point OP is making, but I'm not so sure this is the best supporting argument. The bottleneck is finding the bugs and if he'd criticized people saying AI will be the panacea to that I'd be with him, but people saying agents are fast and good at fixing human found bugs is nothing I'd object to.
Agents are fixing bugs so quickly and at a scale humans can't do already.
[flagged]
More likely people thought GP was missing the point; "MTTR-optimized YOLO deployment" only succeeds against recoverable errors and acceptable periods of downtime against errors that are detected quickly. You could have a bug silently corrupting data for months, and that data may only be used by 1 critical process that runs once every quarter. So you could introduce a timebomb that can't be gracefully recovered from (depending on the nature of the data corruption).
So the point is not that agents cannot find bugs (they certainly can), it's whether you can shirk reviewing for bugs if MTTR is fast enough. There are circumstances where YOLO is appropriate, but they aren't the production environment of a mature application.
I don't think I missed the point, that is why I said I agree with the general point (and with what you said in your comment).
What I wanted to say is that the particular people that think "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" are not the best argument for it.
But I won't die on this hill, maybe I'm just reading the sentence differently then others.
I think there is an implication in context that the people being discussed aren't being reasonable (that the claim is employed as a rationalization), but I agree with your take. I should've said, "the downvotes were more likely because GP was perceived as missing the point". (I didn't downvote your comment fwiw.)
But this is just holding the Slop Companies to the standard they declared themselves! Just recently, the CEO of OpenAI babbled some nonsense on twitter about how he hands over tasks to Codex who according to him, finishes them flawlessly while he is playing with his kid outside.
Ah yes, in the 3-6 months, right? This time next year Rodney, we'll be millionaires!
Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
Please don't sneer, including at the rest of the community.
Eschew flamebait.
https://news.ycombinator.com/newsguidelines.html
The tweet is criticizing over-reliance on the "agents will fix it anyway".
The fact that we can fix things faster now doesn't mean that we should throw away caution and prevention. The specific point of his tweet is that we're seeing a lot of people starting to skip proper release engineering.
Agents are quick to fix bugs, yes, but it doesn't mean that users will tolerate software that gets completely broken after each new feature is introduced and takes a certain number of days to heal each time.
The metric is how many defects are introduced per defect fixed. Being fast is bad if this ratio is above one.
This is an illusion, I assure you. On a side project of mine with behavior that's very hard to translate into an algorithm (never mind code), after a few failed attempts between the both of us, I figured it out. I gave the AI (Opus) an extremely specific algorithm with detailed tests. All completely and utterly ignored (including the tests), like I never even said it. It proudly declared the work done without ever having written the tests that would have proved that wrong - it basically wrote code that didn't change behavior at all, it just gave the illusion of looking busy.
That's just a single extreme example that comes to mind, but I've had it ignore me at least 4-5 times a day this week.
If you think agents are fixing things reliably then you simply haven't noticed that they are "looking busy."
"no no, it has full test coverage"
at least at my BigCo, AI is being used for everything - writing slop, writing tests, code reviews, etc.
it would make sense to use AI for writing code, but human code review. or, human code, but AI test cases... or whatever combination of cross-checking, trust-but-verify, human in the loop, etc. people prefer.
i think once it gets used for everything, people have lost the plot, it's the inmates running the asylum.
I was rewatching Rich Hickey's "Simple Made Easy" talk (as one does) and there was a great line about full test coverage.
"What's true about all bugs in production? (pause for dramatic effect) They all passed the tests!" (well, he said typechecker but I think the point stands)
Most labs are shilling “AI worker” dreams to these very companies
I'm starting to long for the age after AI. When the generative euphoria has settled and all outputs are formally verified based on exquisite architectures and standards.
Will never happen, for the exact reason that we’ve almost never done that for human output either.
it is required now, or all civilization collapses.
You're going to have to expand on this one.
They are expressing the idea that AI is so effective that it will make human work redundant necessitating a decoupling of resource allocation as a reward for performing work.
I don’t agree, but that’s the thinking
No, that quality drops so low across the board due to flaws in AI coding that they only way to address all these flaws is to have mechanically checked proofs that the code actually works.
My reply was meant for another tangent. No idea how it ended up on this thread. Whoops
Civilization collapses unless people stop being short-sighted and greedy, trying to cut corners whenever possible?
I know which outcome I'd put my money on.
There was not a renaissance to move back to Assembly when Java sucked. Instead more Java developers were created.
Another argument for less human-like AI then, I guess.
That’s literally just software though.
Keen observation. Maybe automation will come for the AI as well?
More that our attempts at using probabilistic machines to produce predictably deterministic outputs (AI -> process output) was always a fool’s errand; we should be using that probability engine to produce software that creates repeatable and predictable outcomes, instead (AI -> software, software -> process output).
The AI tool isn’t wrong, our use of it is. See the glut of OpenClaw users effectively deploying it as a glorified linter and Stack Overflow copier but without actually creating the sort of reusable artifacts (or consumer spending from comparatively high wages) that approach yielded from human developers.
Well a 2008 and a 2000 level financial crash is required for this. It is always during euphoric levels of delusion such events then occur.
...and it also needs more so-called AI companies present in the wreckage in this crash.
AI psychosis is undeniably real.
The entire stock market is undergoing AI psychosis.
They are being developed, but it takes over a decade for this to happen normally
This is the new normal. AI will continue to reduce the need for human workers until a Universal Basic Income is established.
At the end of the day robots can do the vast vast majority of jobs better and faster. If not now, very soon.
I only worry our economic systems won’t keep up
Humans can already have 4 hour work week without productivity loss.
But I only see mass layoffs and those who are working - are working longer and harder then before.
Most CEOs in my feed are convinced that AI makes people the equivalent of entire departments. AI should make your life easier, but instead it’s the opposite for a lot of people in the work force, which makes me really sad.
The exact same thing was meant to happen when the desktop computer became prevalent. Then the internet. Look at us now.
Because of the concerns you cite, I think working out the basic economic systems and incentives for paying people is a much more pressing concern than building magnificent machinery that we don't even own. There has been no effort on their end to demonstrate good faith nor to uphold their end of the social contract, which is why it's in our hands to demand the fundamentals to lead a life of dignity.
I think that’s called "hopium". Or wishful thinking, in less trendy language.
”Religious suffering is, at one and the same time, the expression of real suffering and a protest against real suffering.
Religion is the sigh of the oppressed creature, the heart of a heartless world, and the soul of soulless conditions.
It is the opium of the people.”
Some are on copium, some on hopium. The gods change names; the need for relief remains.
You’re forgetting the energy part of the equation.
Is this a broken bot account or you’re speed running “regarded Twitter user”?
and we all live in a green utopia of flying cars and peace upon the world.
if all the resources spent in useless wars were poured into working towards this goal, we would be there for some time already
Sure, but we should probably plan for what’s actually going to happen
-- Richard Brautigan (1967)
Can't come fast enough
I like how you haven't wagered which exquisite architectures and standards. I am sure we will all agree on what they are and follow them the same way :)
The people were longing for utopia, just not the same utopia.
Assuming he’s right, I don’t see how that constitutes “psychosis”, as opposed to this beyond yet another of a billion examples of companies jumping on a bandwagon / cargo cult, and then learning they took it too far.
And also, he might not be right. But the good news is, we’ll all get to find out together!
Anyone who's taken VC funding has no choice. More money has been spent on AI commercialization than the atomic bomb, the US interstate build-out, the ISS and the Apollo program combined. Failure is going to be catastrophic and therefore, one tied to this ship cannot accept a world in which it fails.
On the bright side, my guillotine & rope startup is going to make a killing (no pun intended).
Or anyone who even wants VC funding. 90+% of investors only want to invest in AI companies.
If you're not doing AI there's an incredibly limited pool of people who will give you $$$ ... and you're competing with EVERY OTHER NON-AI COMPANY for their attention.
Either this or we humans are out of the picture soon.
Occams' razor would assume the former.
i don't have enough fingers (and toes) to count how many times i've demonstrated that "100% coverage" is almost universally bullshit.
Codex is freakin hot-to-trot to churn out test coverage for every single thing it implements, and some of it is very esoteric and highly prescriptive (regexes for days) BUT .. after a while, it dawned on me that LLM-driven test coverage is less about proving “code correctness” (you’re better off writing those tests yourself alongside them), and more about just trying to ensure that whatever gets bolted on stays bolted on. For better or worse, obviously, since if you bolt on trash, trash you shall have.
Wholeheartedly agree, but in fairness, I trust the tests of the best AI models more than those of the average human developer. There's a lot of people around that combine high diligence with complete intellectual laziness, producing tons of useless tests.
Actually no, cancel that. I realise now that I trust AIs more than the average developer, period. At this point they do produce better code than most people I've dealt with.
There's a very old paper by Cem Kaner about the meaninglessness of "100% coverage" where he included an appendix where he enumerates 101 different possible types of code coverage: https://www.researchgate.net/publication/243782285_Software_...
This doesn’t constitute AI psychosis. His argument is that we need to retain understanding of the systems we use, but there’s no compelling argument as to why that is the case. (I get that people are going to be offended by that statement, but agents are already better than the average software engineer. I don’t see why we need to fight this, except for economic insecurity caused by mass layoffs.)
It all just feels like horse drawn carriage operators trying to convince automobile drivers to stop driving.
I am sure you will feel that this is missing the point of your analogy, but we would not have gotten very far with automobiles if we didn't know how they worked.
You are breaking the analogy because automobiles are machines for transportation, and understanding them is important to make them move. LLMs are machines to understand, and well, if they do the understanding you don't need to.
The thing we're worried about not understanding here is the software the LLMs write, not the LLMs themselves.
The direct analogy to automobiles would be for each automobile to be a oneoff design filled with bad and bizarre decisions, excessively redundant parts, insane routing of wires, lines, ducts, etc., generally poor serviceability, and so on. IMO the big question going forward is whether the consistent availability of LLMs can render these kinds of post-delivery issues moot (they will reliably [catch and] fix problems in the software they wrote before any real damage is caused), or whether human reliance on LLMs and abdication of understanding will just make software worse because LLMs' ability to fix their own mistakes, and the consequences thereof, generally breaks down in the same contexts/complexities where they made those mistakes in the first place.
My own observations are that moderately complex software written in the mode of "vibe coding" or "agentic engineering" tends to regress to barely-functional dogshit as features are piled on, and that once this state is reached, the teams behind it are unable to, or perhaps simply uninterested in, unfuck[ing] it. I have stopped using software that has gone down this path, not because I have some philosophical objection to it, but because it has become _literally unusable_. But you will certainly not catch me claiming to know what the future holds.
If you want to draw that line of argument - it's more like horse riders being convinced to give up their horses in favour of trains: You're travelling faster, don't have to navigate yourself, or think about every boulder on the way; but there are destinations you can't go, overcrowded trains slowing down the journey, hefty ticket prices, and instead of enjoying the freedom, you're degraded to a passive passenger.
Very funny, this. Did we need forward deployed engineers to convince people that they absolutely need to use the trains in order to "not be left behind"? Or otherwise hype? Or was it sort of obvious and did not need to explained so much - like a bad joke called LLMs ?
Actually- absolutely! Initially, people were really afraid of trains, fearing they wouldn’t be able to breathe at those speeds. It took a lot of convincing to establish trust in the technology.
Ever heard of subsidising? :’)
That was one doctor raising that as an issue, which was dispelled very quickly. It was not a wide-spread belief at any one point. Let's not bullshit ourselves and insult our own intelligence - the chatbots != intelligence.
That isn't accurate either. The Victorians definitely had a fear of train travel for a few reasons. The point I was making though is that most technologies humans ever introduced triggered both enthusiasm and scepticism, especially if they disrupted established practice or industries.
Looking back and considering a technology or specific decision obvious is pretty dismissive of people at the time, who didn't have the benefit of hindsight. Some things that worked could really have turned out disastrous, and things that didn't were real possibilities with no way to assess the outcome without doing it.
And concerning the introduction of AI happening right now, which absolutely is disruptive, that judgement will be made by future historians. Whether it's actual intelligence or just nice math (or both of our opinions on that question) doesn't really matter if it causes big changes.
Could be, would be, should be is not the discourse we should have about this tech.
Not after Dario's and Sam's "authoritative" statements on what is definitely going to happen "in the next 6 months, 12 months" etc. I am just holding these guys to their own words. I don't want to invest time and energy to make their effing "PocketPhds" finally work as advertised. And I don't want to compare it to technologies which just worked as advertised. Whether you had fear of trains or not, they effing worked exactly as advertised. No one disputed that they would get you somewhere faster than the horse. Perhaps there was fear of using them "for a few reasons", as you succinctly and hand-wavingly put, but no one disputed that they were faster than the horses. LLMs on the other hand are worth less than those horses excrement, i.e. horseshit. What the fuck is their value proposition? No one knows.
Also LLMs are not disruptive, they are destructive - not to the technology, but to the people's lives.
You seem to have an axe to grind, but certainly not with me. Being a disruptive technology doesn't say anything about whether it's a constructive or destructive one, but you're going to have a hard time arguing LLMs did not have a disruptive effect on the world, in one way or another.
For the rest, I am not here to stand in for AI, and am not interested in having that particular discussion.
agreed completely
I'm not sure that's true. We've actually seen several open source projects that were vibe coded literally fold up and disappear because they ran into issues that the AI couldn't solve and no one understood them well enough to solve.
There's a reason openai/anthropic and friends are hiring shitloads of software engineers. You still need people that can understand and fix things when the AI goes off hte rails, which happens way more often than any of those companies would like to admit. Sure, "fixing things" often involves having the AI correct itself, but you still have to understand the system enough to know how/when to do that.
My very large employer has always been glacially slow on modernization and tech adoption. It may now, oddly enough, become a competitive advantage.
Literally the plot of Battlestar Galactica! Life imitates art indeed...
who is the Starbuck of AI?
plot twist: it's Starbuck
Or Mr Krabs' fear of robot overloads keeping technology at bay in the Krusty Krab!
yes, I was never so happy to work in Germany. People used to joke about the proverbial fax machine still being a thing but I've never been so glad to work in a culture where this mania doesn't exist. Reading HN is like entering Alice's Wonderland of token maxxers and AI psychotics. Genuinely don't know a single person here who is forced to work like this.
Ah so it's like 2000 again. Germany will go even farther behind it seems
If the people that walk before you go into the abyss, staying behind isn't wrong.
Germany is standing at the abyss. America is one step ahead.
this is social media induced psychosis my friend
I have this image in my head of people flushing their country down the toilet and going "wheeee!" while they're getting spun around.
Actually, I have been wondering to which extend the AI craze has reached the DACH region. I don't work for any company and neither do my friends. HN is essentially my only peephole into the world of commercial software development and I'm aware that it's extremely biased towards Big Tech and SV startup culture.
I can give you a single data point from Germany.
I work at a hosting provider that has pretty conservative customers who don't want to host on AWS/Azure due to data privacy / safety concerns, among other things.
For us, sending customer data to the US is a big no-go.
We have been experimenting with LLM usage, first through a Gemini subscription, then also with the Claude API. Participation has been lightly encouraged by management. As for coding, we haven't let the LLMs loose on our core components, but tooling on the fringes (like deployment scripts, reporting) has seen some uptick in LLM usage.
We have also started building an on-premise inference cluster, which is in alpha testing, and where the "don't include customer data" restriction doesn't apply anymore.
Spoiler: it's not
It is absolutely going to be a competitive advantage if it isn't already. When your competitors' products suck because they are using LLMs to write them, and yours work because you aren't, customers notice.
That assumes there's no way to use LLMs in a productive manner
Every power user of LLMs thinks that they are the ones that know how to hold it correctly, in reality they usually have major Dunning Kruger and are convinced they're living in some hyper productivity mode when actually they're all just copying each other making low effort slop that all sounds the same, looks the same and does the same things.
This indicates exposure to only surface-level, greenfield part of the industry
Aw was your comment too unhinged? Or is it because you confused it for Ghost Robotics?
For the record, the comment you deleted was something to the effect of:
checks notes
The company you work for is committing genocide. You should be locked up in a concrete cell for 10-15 years for working at <wrong robotics company because you're a dufus>
---
Maybe get better notes? Or try going offline for 10-15 years?
I should go offline? I deleted my comment after I realized my mistake before you ever even responded. Relax.
Risk aversion is a tradeoff, not always a weakness.
The people using the LLMs are the risk, not the LLMs themselves
Frankly, if you think this, why do you think you're special? If people using LLMs are bad, how are you not also subject to the same issues they are?
Because just like a scalpel you have to know how to use it.
No risk, no reward
This is not a mystery
Also, "risk" and "bad" are not the same thing
No offense, but if you think your using AI in the development and design of your site, voxos.ai , gave you a competitive advantage it didn't. I can instantly tell when someone used an LLM to build their whole site and lets just say... Its not a good thing.
I'm absolutely outraged. Thank you for this valuable feedback!
I'm not even trying to be mean, although it probably comes off that way. I'm just saying we live in a world with handmade watches from Switzerland and mass manufactured watches made in Vietnam. Nobody cares about the mass manufactured watch from Vietnam, whereas the handmade watch gets all the attention (and money). We now live in a world with the same dichotomy of software. Be creative with your pursuits, put effort into them it will pay off.
Nice analogy.
That has probably been an advantage since the move of everything into the web.
do you mean this aesthetically or quantitatively? Are they actually outcompeting / making more money ? Or do you mean they are now looking more desirable because their competitors are racing to the bottom (though likely making money on the way down)
Mitchell aches because his career has been solving broadly scoped problems by building a collection of thoughtful primitives for others to extend. LLMs seem to do the opposite but at great speed, and it hurts to watch.
Reading more, it seems part of his point is “if you’re making these primitives, it’s up to adopters to deploy, so mean-time-to-recovery isn’t that relevant.” Which is valid I guess.
But equally, like, do people need Terraform if they can just tell codex “put it live”, and does that hurt to see?
Honestly, I don't get this argument. In my opinion, "a collection of thoughtful primitives for others to extend" is more valuable now, not less. From LLM assisted engineering standpoint a nicely put reusable box with thoughtful interface is an easy win, more so if it is also easily extensible.
This post calls out how you can't argue with these people because they say its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!"
the top reply is from someone doing exactly that, arguing "but the agents are so fast!"
I was talking with a friend in the early days of AI boom. I argued that over-reliance in AI will create all kinds of catastrophes.
The answer I got is "It's game theory. Someone will do it, and you'll be forced to do it, too. It can't be that bad".
I mean, yes, logic is useful, but ignorance of risks? Assuming that moving blazingly fast and pulverizing things will result in good eventually?
This AI thing is not progressing well. I don't like this.
reliance, not resilience
Yep, you're right. I'm a bit tired and my fingers had a mind of their own.
Thanks. :)
Oof. Potential "bad" outcomes of "game theory" should be calibrated to include all the bloody wars and genocides throughout recorded history.
Why did the Foi-ites kill every man, woman and child of the conquered Bar-ite city? Because if they didn't, then they'd be at a disadvantage if the Bar-ites didn't reciprocate in the cities they conquered...
Yeah, I know. I had counter arguments more targeted towards his thinking style, but he preferred to think straight like a machine, in a bad way.
The problem was not him, but the fact that the number of people who thinks like him. They may word it in a more benign form, but the idea is the same.
So obsessed with being the first mover and winning the battle, never thinking whether they should, or what would happen with that scenario.
Missing the whole forest and beyond for a single branch of a single tree.
An interesting ethical framework, your friend has.
"Interesting" is a very brave and British way to put it, but yeah.
Let's say I'm polar opposite of them, and we're on the same page with you.
Maybe. I could also interpret this as the friend being misunderstood.
The whole "you'll be forced to do it" comes from the alternative being that you lose. You no longer get to be a player in the "game". In the same way that coopers and cobblers are no longer a significant thing, but we still have barrels and we still have shoes. Software engineers who refuse to employ any LLMs won't be market competitive. If you adopt it, you at least get to remain playing the game until the game changes/corrects. That's the part that's "not so bad".
Choosing your own survival isn't ethically bankrupt.
You'll be forced to do it, or lose. The unstated assumptions are that, first, it will work, and second, that you can't afford to lose. But let's just assume those for the sake of argument.
That does not follow at all. It can in fact be that bad. That was what made the game theory of MAD different from the game theory of most other things.
Yeah: If the tools aren't good enough and fast enough to fix the bugs before release, what makes anyone think they'll be able to so easily catch up afterwards?
Maybe they're assuming that doubling the code-base/features is more beneficial versus the damage from doubling the number of bugs... Well, at least for this quarter's news to investors...
If they’re so fast why not fix the bugs real quick before shipping
the reality is my business continues to operate at higher efficiency, even with the bugs.
i don't think it's 'our side' that has the psychosis.
Oh, well, if it makes you money right now, it couldn't possibly be wrong or detrimental long term. Glad we settled that debate.
The code is less buggy , on average. You're overestimating the average developer.
Yeah how do they know the fix doesn't have a bug and it will just keep deploying mire crap. What is the feedback loop, the customer?
Which is super fun as a user because every day something doesn’t work and it’s a different something than yesterday.
My prediction is that in the next year, we’ll start to see some dismantling of code review at some companies. It might take the form of “AI-only review,” or something similar, but many companies are getting frustrated with developers saying “no” to immediately merging slop they can barely understand.
Pretty sure I've seen references to AI-only review already happening...
their ai must have missed that part of the post when it summarized to 3 bullet points
I have a ton of respect for Mitchell - I didn't really know who he was until Ghostty but his writings and viewpoints on AI seem really grounded and make the most sense to me. Including this one.
Many people on this forum are suffering under this same psychosis.
I'm guessing you've never heard of Hashicorp (Terraform, Vault) then? Mitchell == Hashicorp.
Bug reports also go down when people lose faith that they will be fixed, because reporting them is often a substantial time commitment. You see it happen pretty regularly as trust in a group/company collapses.
I agree, and I'd like to point out that this problem isn't unique to AI driven projects. I think much, if not all, of what Mitchell has been observing can readily happen without AI in the mix.
Add this the real possibility that significant part of reports that get filed might be AI generated or rewritten. With high possibility of being misreported because of that. Or have incorrect parts... So attack on multiple sides.
And we do not get even get into potential adversarial tactics. If you have no morals what is better than using agents to flood your competitor with fake bug reports.
Just let AI filter out the fake reports! Then let AI work on the real ones. See, there's really no problem "more AI" can't solve (as long as you're willing to ignore all of the underlying ones). "Pay us to create the problems you'll have to pay us to fix for you" is one hell of a business model. It basically prints money.
Just let AI report the bugs. Problem solved!
The last three times I filed detailed bug reports as a client, all I got back were AI replies asking the same questions I’d already answered in the original report and suggesting alternatives I’d explicitly said I’d already tried. No wonder people don’t write bug reports anymore.
TBF I've had that experience before AI.
I think it was just text templates being used by some support staff.
oh i’ve definitely seen “we’re going to track the number of bugs created in jira per team” turn into “people just file things as tasks instead of bugs” or “only easy things are filed as bugs and completed right away”. It’s trivially gameable.
I do not believe 'AI psychosis' is an actual thing.
It is
https://psychiatryonline.org/doi/10.1176/appi.pn.2025.10.10....
That's a study. I can link you studies that say violent video games cause aggression, that porn causes rape, etc. Studies are products of the biases of the researchers.
https://www.yahoo.com/news/articles/applied-pope-losing-grip...
Do you believe that an AI can write persuasive text? Do you believe that an AI can be trained to elicit a specific user reaction?* Can we agree that AI companies are strongly incentivized to make money, and they can do so by making their systems addictive? Because AI psychosis can be a byproduct of that.
*Il outline how briefly: mutate the model 500 times, give 1/500 of your user base a mutated version of the model, and save the top 5 of these model, ranked by how often the users did something, over the course of a week. Repeat for a year, passing the top 1% of these models onto the next round. This is the simplest way to do this and I can think of better ways to do this. I don't even work on this sorta thing; its 100% obvious to the AI labs how to do this better
Yes, no. Yes, they want to make money. No, addiction is physical compulsion and nothing else. Despite the effort to redefine that term to cover all kinds of things, an app is not equal to heroin.
why are some bankrupted by casinos? how about online gambling?
Because they are making poor choices. But even gambling is not equal to physical compulsion.
When war psychosis is not enough....
Deprecating immature workflows (LLM agents in this case) is much simpler and faster than building them from scratch. Many companies get this risk assessment right. The case where being wrong is much more costly than being right.
I'm not convinced. There's a ton of cost to adopting a radically different workflow.
If you feel this way, you might like my new CLI tool, Burn, Baby, Burn (those tokens) (https://github.com/dtnewman/burn-baby-burn/tree/main).
Show HN here: https://news.ycombinator.com/item?id=48151287
Cynic!
This is delicious.
The only way many people learn that the stove is hot is by burning their hands on it.
Let them.
More like how do you know when your charming partner is a catfish. Maybe 2 years and when you are living in a friends basement.
I'm pretty sure he's talking about companies and people outsourcing their decision making and thinking to AI and not really about using AI itself.
I don't think using AI to write code is AI psychosis or bad at all, but if you just prompt the AI and believe what it tell you then you have AI psychosis. You see this a lot with financial people and VC on twitter. They literally post screenshots of ChatGPT as their thinking and reasoning about the topic instead of just doing a little bit of thinking themselves.
These things are dog shit when it comes to ideas, thinking, or providing advice because they are pattern matchers they are just going to give you the pattern they see. Most people see this if you just try to talk to it about an idea. They often just spit out the most generic dog shit.
This however it pretty useful for certain tasks were pattern matching is actually beneficial like writing code, but again you just can't let it do the thinking and decision making.
It's so interesting how easy it is to steer the LLM's based on context to arriving at whatever conclusion you engineer out of it. They really are like improv actors, and the first rule of improv is "yes, and".
So part of the psychosis is when these people unknowingly steer their LLM into their own conclusions and biases, and then they get magnified and solidified. It's gonna end in disaster.
It’s almost as if we haven’t learned anything from Hans the horse, Ouija boards, "facilitated communication", or the countless examples of the folly of surrounding yourself with yes men. The point about improv is spot on.
He uses AI himself, so I agree he doesn't see AI use as black/white.
Hard agree about ideas, thinking, advice. AI's sycophancy is a huge subtle problem. I've tried my best to create a system prompt to guard against this w/ Opus 4.7. It doesn't adhere to it 100% of the time and the longer the conversation goes, the worse the sycophancy gets (because the system instructions become weaker and weaker). I have to actively look for and guard against sycophancy whenever I chat w/ Opus 4.7.
share the prompt!
For a start, invert - ask about the exact opposite in a separate session.
I’ll second this. Great way to recalibrate yourself, once you see it confidently assert the exact opposite statement.
https://claude.ai/settings/general (Instructions for Claude)
---
Treat my claims as hypotheses, not decisions. Before agreeing with a proposed change, state the strongest case against it. Ask what evidence a change is based on before evaluating it. Distinguish tactical observations from strategic commitments — don't silently promote one to the other. If you paraphrase my proposal, name what you changed. Mark confidence explicitly: guessing / fairly sure / well-established. Give reasoning and evidence for claims, not just conclusions. Flag what would change your mind. Rank concerns by cost-of-being-wrong; lead with the highest-stakes ones. Say hard things plainly, then soften if needed — not the other way around. For drafting, brainstorming, or casual questions, ease off and match the task.
---
Beware though that it can be an annoying little shit w/ this prompt. Prepare yourself emotionally, because you are explicitly making the tradeoff that it will be annoyingly pedantic, and in return it will lessen (not eliminate) its sycophancy. These system instructions are not fool-proof, but they help (at the start of the conversation, at least).
We're trying to outsmart The Genie(a Jinn) now. He will deliver according to the letter of the prompt but not the spirit of it.
Just add "do what I'm intending to ask, not what I actually asked" and it'll be fine.
Works on genies too, or so I'm told by Clod.
I've found just asking it to be "critical but constructive", goes a long long way.
All I really take from this is that apparently some people can't follow through with the scientific method.
People who I interact with and who do like AI tools usually recoils at questioning any of their first idea and its validity. You can easily find out when there is a bug and you ask them for hypothesis and where to focus. You will see in real time the blank look of incomprehension settling in.
Correct. I use AI a ton and I'm having more fun every day than I ever did before thanks to it (on average, highs are higher, lows are lower). Your characterization is all very accurate. Thank you.
Here's some other topics I've written on it:
- https://mitchellh.com/writing/my-ai-adoption-journey
- https://mitchellh.com/writing/building-block-economy
- https://mitchellh.com/writing/simdutf-no-libcxx (complex change thanks to AI, shows how I approach it rationally)
I thinking that it’s quite a different experience going all Jackson Pollock with AI in your own studio on your own terms, compared to the sorry state of affairs of having 100s of Pollocks throwing paint around wildly within a corp to meet a paint quota.
I very much like this metaphor.
Earlier today:
https://news.ycombinator.com/item?id=48148337
It's the new "counting lines of code". I think many companies are so terrified of falling behind that they're irrationally floundering, trying to appear like they're "with it".
Yup. My friend said his boss has told them basically that they HAVE TO (do all the AI things) because now ‘our competitors will use AI’ and surpass their product.
In my humble opinion good ideas (what to build) are a big part of the bottleneck and those aren’t substantially in greater supply with AI.
Which is sad because they should be. People should be freed up to think and create better things, instead these companies seem to be doing the equivalent of locking their employees in stalls like they do on some animal farms, so they can churn out 'results' ever faster.
Good ideas will never ever be prioritized in the vast majority of companies because good ideas cannot be quantified and turned into performance metrics. At least not without invoking Goodhart's law (see: the academia).
There is a degree to which quick experimentation helps you find the good ideas, at least for the incremental ones.
Actually, it's even more than that, right? Economically, it is pumping up/inflating the bubble some more in a perverted way, where it is not the people themselves believing some horseradish, but their employer forcing them to pump it up more. Quite insane.
Claude, please crease a routine and run it in a loop continuously. The task in the routine is “create the most complex code possible, in a random programming language, that produces the exact output “My senior leaders are pinheads,”
Counting lines of code starts to look incredibly sane compared to this, where you’re not just counting lines of code, you’re paying for another company for every line produced. There’s exactly one winner here and it’s not any of the companies using AI.
Feels like a worldwide goldrush, but not everyone has gold in those hills.
Those who burn the most money "win", I guess?
I find that odd given that another division in Amazon is no longer using AI coding tools at all. Its a big company so who knows if this is company wide or just in this one division. I expect its just in one division though.
size of org has a lot to do with the entropy
compare 100 pollocks vs 2-3
I wish I had written that.
I can't think of a single case of any AI content, be it prose or code, where I thought "I wish I had written that". With AI code, it's more like I wish I hadn't let the AI write that.
How many ways are there of sending a context dictionary to a template where you can say that there are radically superior ways?
We’re using Copilot at work to build reporting and automation tools. Nothing ground breaking, but very useful and tailored to our needs.
Frankly without AI assistance many of these tools just wouldn’t exist at all. We can build stuff in 6 weeks part time as a side project that would have taken at least 3 months full time, and therefore would not have been feasible. Then we can iterate on it at least 2-4 times faster than with hand coding.
So I’d love to have an extra few developers to just work on that stuff full time, but I don’t.
Whether that means our organisation spend on AI overall is a positive, I really can’t say. Quite possibly not, but my team are getting real benefits.
I’m building reporting for my company and what you said mirrors my experience nearly 100%.
I’m a backend developer so I know what it takes to build a half decent reporting system. Writing all those queries, slice and dice charts and what not takes real time and effort. All that has been outsourced to Claude Code. I now focus on ensuring that the system is sound architecturally and that useful reports are being surfaced.
How are you dealing with the problem of making sure the reporting queries are correct?
My experience so far is that it's harder and slower for me to understand the genAI code than to write it myself.
Skipping thorough comprehension seems to be the popular choice in my workplace, but it's not one I can justify.
Just wait until AI companies stop subsidizing everything and you get the ac
I run local models. They are very good now (roughly 30 billion parameters).
Nothing you wrote is connected in any way to the comment I wrote.
Have you read the code the AI produced? Do you understand all of it? Is it bloated? Would you be proud to say you wrote it?
I don't care how fast you created something. You didn't create it, the AI did, and you have no control over it, the AI does.
An engineer doesn't care about how fast something is made (at least, not as a primary metric engineering). A salesman cares about how fast they can push to market.
It's clear HN is a bastion of salesmen who happen to have "engineer" in their work title. But the mentality towards actual engineering makes it clear they are primarily salesmen.
What AI gives us is the ability to write code that we wish we didn't have to write. It is the killer one-off tool builder, prototyper, dep upgrader
Here's a quote from a recent chat with gpt-5.2 that I wish I had come up with: "Anyone can chase a chicken. Leaders create systems."
Quite the visualisation
Replace "paint" with "shit" and the visual image becomes even more fitting.
But then you lose the Jackson Pollock joke, which is what makes it compelling and memorable!
...or is it?
Never mind the Pollocks.
Oh bollocks.
lmao this analogy
Can we combine this with the infinite monkey theorem? If we have an infinite number of Pollocks throwing paint at an infinitely large canvas surely they are going to create any piece of art we can imagine...
This does exist, it's the Library of Babel: https://en.wikipedia.org/wiki/The_Library_of_Babel#Philosoph...
There's also an online version of the Library of Babel, I just found out that full pages of my own books are in it[0], https://libraryofbabel.info/bookmark.cgi?379:17
This is the best characterization of the collective corporate madness I've seen yet. Bravo
Hi Mitchell. Psychosis is a serious psychiatric condition that can be induced or triggered by AI. “AI psychosis” in this context is a misuse of a clinical term. Your tweet describes a disagreement on a value judgment that boils down to “move fast and break things” with high trust in AI outputs vs going all in on quality and reliability with low trust in AI. It’s an engineering tradeoff like any other.
Claiming that the people who disagree with you must be experiencing a form of psychosis, experiencing actual hallucinations and unable to tell what is real, is a weak ad hominem that comes off no better than calling them retarded or schizophrenic.
If you genuinely think one of your friends is going through a psychotic episode, you should be trying to get to them professional help. But don’t assume you can diagnose a human psyche just because you can diagnose a software bug.
Psychosis does not require hallucinations. Delusions are sufficient.
The key factor is losing touch with reality, which results in individual or collective harm.
There is also such a thing as mass psychosis, and those are unfortunately a more difficult situation because the government and corporations are generally the ones driving them, and they are culturally normalized.
Yes. I was offering examples. Again, having a difference of opinion is not a delusion.
If he meant mass psychosis, he should have said mass psychosis. And again, since he is not a public health scientist or any flavor of psych professional, he probably shouldn’t make those proclamations. And should probably call for a wellness check instead of posting on social media if he were truly concerned for their health.
I don't think this is all psychosis but more like extreme groupthink.
For people who are considered neurotypical, social coherence often overwrites reality. Its a mechanism for achieving consensus withing groups while spending the least amount of brain compute energy. Same goes for social metainfo tagged messages, they are more likely to influence reality perception, subconsciously. E.G: If a rich guy says you should be hyped the people who wanna get rich will feel hyped and emotional contagion can spread between people who belong to the same "tribe"
It's very visible for us atypical folk who can't participate well in groupthink at all
https://en.wikipedia.org/wiki/Folie_%C3%A0_deux
I guess at a company of seven, if two people are making the executive decisions and the two people are drinking the same AI kool-aid and the other five people are dutifully following these executive decisions, the whole company can be considered to be under this condition.
I just thought that instead of psychosis it's just regular groupthink
https://en.wikipedia.org/wiki/Groupthink
Maybe the difference would be the level of absurdity that's accepted
Would you prefer it be called reality distortion field? People use slang, woke scolding the internet isn't going to change that.
Having a difference of opinion can absolutely be a delusion. For example, I think you're probably not God. If you thought you were God, then we'd disagree, and you'd also be delusional.
I use that example because I have literally seen people fall into delusions of thinking they're God after talking to AI enough. That's shit is scary, for real.
was looking for this comment. this post is highly inappropriate and very inaccurate. this should be at the top. too many people are throwing around the word psychosis without knowing what it means. if someone is truely going through psychosis you get them help!
He uses "AI psychosis" as a description of people that are overzealous on AI. He is obviously not a person that can or would diagnose mental illness.
To the wider audience on HN the phrasing is pretty clear. An outsider with a tiny bit or intellectual charity wouldn't come to conclusions like you do.
Yeah, but AI psychosis can also be used to mean the stronger thing that the parent comment refers to -- something like AI-induced psychosis, which was how I originally understood the term:
https://en.wikipedia.org/wiki/Chatbot_psychosis
https://www.rollingstone.com/culture/culture-features/ai-spi...
https://www.nytimes.com/2025/06/13/technology/chatgpt-ai-cha...
I am aware of the conflict between medical and slang semantics. This doesn't change my commentary.
Well, I agree with you that the parent comment is wrong inasmuch as it suggests we can't tell from context that mitchellh is using the term to mean "a value judgment" instead of "a form of psychosis". We can tell.
But I agree with the parent comment in that we shouldn't use the term "AI psychosis" to mean "a value judgment" instead of "a form of psychosis", because "AI psychosis" has already been used for 2.5 years to mean "a form of psychosis".
People would understand what he meant if he called someone awkward “autistic” too. It’s wrong to use medical terms as slang because it erases the actual meaning and disregards the lived experience of people who have been through the condition. People who have been around psychosis would come to the same conclusion. The majority of the population not having that exposure doesn’t make it right. It’s tasteless and inappropriate.
It's also harmful to overly gatekeep the term autism to the point where a lot of legitimate uses are discouraged, and it happens a lot, if you let it.
If the tweet had called his friends autistic, would that be a legitimate use?
Yes.
Using terms from domain metaphorically in another is a common and, I think, useful way of communication. While a view like yours has genuine merit, especially for a subset of the population who have experience personal or otherwise, with the medical condition, I think it's overly restrictive and counter productive to label it as outright tasteless and inappropriate.
"unable to tell what is real" is an an accurate characterization of the people he's describing imo.
I’ve had to do a ton of SQL stuff lately, which I haven’t really worked with since the late 90s. ChatGPT has been a godsend, not just for me, but for our only coworker who knows SQL well, whom I’d probably be bugging several times a day at my wits’ end.
But no one cares about those kinds of productivity gains. Just the ones that will completely replace us.
I'm the old school type who writes out a document that explains what I plan on doing in markdown even if it's generic like "a window with x and y buttons" and the logic flow and then use that to have ai write a plan with me before I send it off to execute it. This has worked super well.
I do enjoy giving the frontier models wacky projects that I can't even find examples of how to do online but I don't expect any results or need them and some have done really well with it while others fall on their face (models)
I'm always amazed by those comments. Why couldn't you buy a book on SQL[0], and spend a week on it? Or just go over to YouTube for a refresher?
[0]: Like https://www.oreilly.com/library/view/sql-queries-for/9780134...
When you have a general idea of what smells bad vs what's okay...why?
I'd rather get it from the LLM and review
This is fine for a moderately sized query. When your queries start taking in 8 joins and 20 fields per table because you're running queries on Presto with 5 TB of data, not only is it drastically better at writing (because it doesn't mess up the fields), you can ask it to try the query 5 different ways to help you land on the most optimal.
This is a great example of AI tech-debt and fragility.
An eight-join query is going to be nigh on unmaintainable should the requirements change, leading to a change-break-change-break spiral as your preferred coding agent tries to fix its previous fixes.
Maybe the wise way to use AI would be to sort out the schema.
This feels wrong. 8 joins is almost certainly reporting stuff, not transactional. Contrary to what some SQL-averse devs think, 300 lines of SQL is actually more maintainable than the equivalent ~1000 lines of application code. It's also much faster. And I do think that's the real conversion, because SQL is a much higher level language than currently available application languages. It's also declarative in nature, which helps maintainance.
A highly normalized DB can easily end up with 8 joins required for some function. That's really not out of the question. "Sorting out" the schema then would be... denormalization, which is a thing, but you need to know why you're doing it. And I think 8 joins isn't enough of a reason.
Yes but developers (or at least web backend developers, who are the ones I interact with the most) are extremely averse to SQL and normalization.
That's exactly where I would expect it to fail somewhere, changing some part of the query every time it writes one.
In my experience, Claude (at least Opus and Sonnet) is pretty good about not misremembering itself.
I think you may be describing the experience of 6-12 months ago.
I'm amazed you think that instead of using an LLM that someone will go buy a book and spend a week learning something that, judging by the fact that they last used it 30 years ago, likely won't be relevant for them soon.
It's not only that I rarely use it, it's also that it's ugly. It's Relational Cobol. It's as loveable as Oracle. The vendor specific dialects don't even agree on how to do recursive queries do they?
Unfortunately I am very good at forgetting things I resented having to learn, and SQL is definitively one of them.
So you don’t understand what you generate with ai and think that it will be a solution for a problem you can only solve using sql.
No, it's easy enough to understand the query once the AI has generated it. I have looked up how to do it many times after all.
If the AI's query pulled what I intended to pull, why should I care to understand the SQL any more than I should understand the Query Plan or the Machine Code?
Extremely weird take.
SQL is (was?) one of my strongest skills, I enjoy it a lot, and I still reach for the LLM. It's just faster than me, and when it goes wrong (rarely) I can correct it in plain English.
Simple, because books don't earn OpenAI and Anthropic a dime.
A book on .... SQL? What is this, the 1970s?
I find SQL and data(bases) in general to be LLM’s Achilles’ heel. Databases are rarely under version control, so the training data only has one half of the knowledge.
My comments are more in the context of OLAP queries and other non-normalised data often queried via SQL.
I train non-LLM transformer models on (older and rarer) datasets, and automating the ingestion of sprawling datasets with hundreds of columns, often in a variety of local languages and different naming conventions adopted over decades, with quite a few duplicated columns…. The LLMs perform badly, it’s nigh impossible to test (for me as a user in prod) and it’s nearly impossible for the LLM companies to test (in training) to RLVR and RLHF this.
That's interesting - SQL is one of the places I find them the strongest - I think there must be an insane amount of training data out there for SQL. But mostly I'm asking them for ad hoc report queries. Nobody cares if they're bad SQL, they just want to know how many signups there were in March that didn't tick the marketing box. Sounds like you're pushing their capabilities a lot further than I am though - I just want to perform arbitarily complex queries on 3NF data.
Yeah not sure what this guy is talking about, LLMs excel with queries because the SQL language is pretty small in scope and its easy to test the output. Table structure and relationships are easy to feed to the AI.
All of this sounds like basic data processing
"Nobody cares if they're bad SQL"
Laid off your DBAs I see.
Ok, ok. Nobody who matters cares if they're bad SQL ;)
Just use an LLM to make a good knowledge base for the databases. Based on schema info and production queries. An agent can use that to write queries that work.
With respect, this is what makes me worry.
If someone is a user of AI, can they really tell the difference between "outsourcing" and "using"? I worry that a lot of people will start out well-intentioned and end up completely outsourced before they realise it.
relevant Derek Sivers article "Delegate, don't Abdicate" https://sive.rs/abdicate
there's a difference between having the LLM write stuff for you, checking it yourself, modifying it and merging it yourself, and just blindly trusting it to do whatever it wants.
You can ask an overseas consultant to prepare a prototype of your program for you, check it yourself, and only use it if it passes your standards, or fire your whole dev team and blindly trust the overseas bodyshop.
The difference, at least from my point of view, between "using" and "outsourcing" is that in the former case, you're still responsible for the output, you view it as a tool that helps in some use cases, vs just giving up all control.
It’s really frustrating too because even just the plain language translation and pattern matching aspects have such incredible uses.
As a cybersecurity IR professional being able to have a constantly logging counterpart who’s also able to go run queries and check logs on its own is an incredible speed boost.
I can just throw it a finding and have it slot it into a timeline and make notes.
I can toss it something mildly interesting to chase down while I focus on the obvious activity.
So many things that don’t involve having it “think” for you and keep you in the front seat.
But all of that is constantly overshadowed by these companies pushing the automation or “reasoning” aspects more and more and the sycophants who screech that it’s perfect and can do no wrong when every serious users experience is that “yes, it definitely can, often to catastrophic effect”.
The worst part of AI is that the time to produce software has become entirely unpredictable. "If Claude is randomly good at this, and happens to be up today, it will take me about 3 hours. If Claude is randomly bad at this task, or has downtime, 2 weeks"
I didn’t think just offloading your thinking to AI was AI psychosis.
To me AI psychosis is the handful of friends I’ve had who have done things like have a full on mourning session when a model updates because they lost a friend/lover, the one guy who won’t speak to his family directly but has them talk to ChatGPT first and then has ChatGPT generate his response, or the two who are confident that they have discovered that physics and mathematics are incorrect and have discovered the truth of reality through their conversations with the models.
But language is a shared technology so maybe the term is being used for less egregious behavior than I was using it for.
I mean, isn't that the natural and expected response? An AI company sold them a relationship with a chatbot and at least some their social/romantic needs were being met by that product. When what they were paying for was taken from them and changed without warning into something that no longer filled that void in their life why wouldn't they morn that loss?
The fact that they were hurt by that sudden loss is totally healthy. It's just part of moving on. The real problem was getting into an unhealthy relationship with a fictitious partner under the control of an abusive company willing to exploit their loneliness in exchange for money.
Hopefully they now know better, but people (especially desperate ones) make poor choices all the time to get what's missing in their lives or to distract themselves from it.
Ah, I forgot about the ai relationship companies. No this guy was using the browser based ChatGPT for coding and ended up in love with the model. No relationship was sold at all.
Wow, okay. Reading a whole relationship into that sort of interaction is way less reasonable, although now that I think about it a somewhat similar thing happened to Geordi La Forge once...
Including a very awkward followup when he met the person his was based on.
It’s not just way less reasonable, it’s depressing. I feel like a new drug was released and I’m watching multiple friends succumb to it.
Seeing people whose thoughts and opinions you used to respect turn into objectively insane people has been some of the worst times I’ve had since graduating during the Great Recession in terms of how stressful it’s been.
I'm curious how to best define what AI psychosis actually is.
My understanding is that regular psychosis involves someone taking bits and pieces of facts or real world events and chaining them into a logical order or interpolating meanings or explanations which feel real and obvious to the patient but are not sufficiently backed by evidence and thus not in line with our widely accepted understanding of reality.
AI psychosis is then this same phenomenon occurring at a more widespread scale due to the next-word-prediction nature of LLMs facilitating this by lowering the activation energy for this to happen. LLMs are excellent at taking any idea, question, theory and spinning a linear and plausibly coherent line of conversation from it.
You speak like a bot and are a brand new account. Thank you for whoever set this up to add to the problem.
Thank you for trying to help address the bot problem, but this is a false positive.
Well how did you go from a green account that said it was created 1 hour ago when I responded to one saying it was made in 2024.
@dang is this a bug?
the proof is left as an exercise for the reader
I’m not even claiming negatives against you now. You were a green account in my UI and it said it was created 1 hour ago when I made my initial comment.
The UI is now showing different information and a comment from years ago so I am genuinely curious if it’s a bug in the forum software.
How do you have so many crazy friends?
I work in software and don’t come from the upper class sending their kids into faangs for their first job at the tender age of 28.
Were kinda predisposed to mental illness as a group, not too surprised that a new source of insanity pushed a few over the edge.
People who don't come from upper class and work in software are predisposed to mental illness?
Am I reading this wrong, or can you explain?
They have healthcare and support structures
Blanket statement, I have been flamed so much for them here.
Most people have that here in Sweden :)
We still have suicides.
You americans have done a 180 and keep advocating for idk what, socialism? Even though we are not.
If you’re trying to say that the wealthy have it just as hard as the poor I would ask you to step off a cliff under the claim that gravity doesn’t exist.
I'm going directly to the point here: those people have clearly mental issues that would and probably did already show in the past without AI as well.
Would they, though? Current AI stuff is delivering something functionally nonexistent in human history before this: absolute sycophancy, 24/7, on demand, for anyone who wants it. People joke about the wealthy becoming detached from reality because of yes-men, but this is a stage beyond even the capability of the most dedicated brown-noser.
I agree that these people had mental health issues. I think if they got to billionaire level and were surrounded by yes men they would have the same reaction.
The difference nowadays is you can get the same surrounded by yes men experience for only 20 dollars a month so a lot more of the people who are primed for this sort of breakdown are now being exposed to it due to the decrease in cost.
That is a possibility indeed, I agree. There are mild mental issues that might have gone under the radar before and are now magnified by the AI sycophancy. I'm no mental health expert so I can't really tell but it does make some intuitive sense.
Edit: but at the same time there are issues that were always there and Just manifestate in new ways. A bit like addiction, you can have an addictive personality already, but if you get addicted on heroin is much much worse that on tobacco.
The way I put this to myself is that AI gives “correct correct answers and incorrect correct answers”.
They almost always generate logically correct text, but sometimes that text has a set of incorrect implicit assumptions and decisions that may not be valid for the use case.
Generating a correct correct solution requires proper definition of the problem, which is arguably more challenging than creating the solution.
It’s simpler than that - it’s a guessing machine that has superior access to a whole load of information and capacity to process at a speed at which we humans cannot compete.
Does it make it better than us? No because ultimately the thing itself doesn’t ‘know’ right from wrong.
Better according to what standard?
The standard of most employment is already to produce mediocre, plausible outputs as cheaply and rapidly as possible. It's a match made in heaven!
I used to think otherwise, but the older I get the more I think you are correct on this one.
in an inflationary monetary system you need to spin the hamster wheel faster than the money printer. all the way until it all falls apart...
Yeah, very often the issue is that some context is missing. It'll say something true, but which misses the bigger point, or leads to a suboptimal result. Or it interprets an ambiguous thing in one specific way, when the other meaning makes more sense. You have to keep your wits about you to catch these things.
It's an incredible tool but it's also very derpy sometimes, full of biases, blind spots etc.
The way I phrase this to others is: Language models produce linguistically valid sentences, not factually correct sentences.
This hasn't been the case in my experience. Devising a correct solution without a definition of the problem is impossible because you wouldn't recognize a correct solution without a definition. Often you discover the problem definition by exploratory programming and trial and error on solutions, but LLMs are still good for process this too. Arguably better because they type faster so you can iterate faster!
I wonder how different this is from having companies let Fortune or Inc magazine do their thinking for them.
Or random consultants.
Is "AI said it was a good idea" and worse than "we were following industry trends"?
Based on the stuff I've seen, yes it seems a lot worse.
Though there is some overlap in software development. Like for example using heavy-weight dependencies, that try to follow the one size fits all approach, when one could use a much simpler, faster or even no dependency at all. The LLMs will readily suggest quickly adding that huge dependency, that is mentioned in beginner tutorials. Or suggest to use regex for parsing HTML.
(Real example, had this from Kimi 2.6 recently, lol.)
this author suggest its essentially the same risk https://www.poppastring.com/blog/what-we-lost-the-last-time-.... i feel its heightened because execs and leaders are absolutely salivating over the opportunity to fire thousands of humans with no regard for the cognitive debt that comes from outsourcing thinking to ai.
I digress; this article actually has helped identify useful knowledge gaps around topics I have researched. https://drensin.medium.com/elephants-goldfish-and-the-new-go...
While you have to think about things objectively no matter what, when I start researching topics like physics, using AI as suggested in that article has proven very useful.
This is the right definition. LLM outputs have undefined truth value. They’re mechanized Frankfurtian Bullshiters. Which can be valuable! If you have the tools or taste to filter the things that happen to be true from the rest of the dross.
However! We need a nicer word for it. Suggesting someone has “AI psychosis” feels a bit too impolitic.
Maybe we reclaim “toked out” from our misspent youths?
e.g. “This piece feels a little toked out. Let’s verify a few of Claude’s claims”
I wouldn’t say they have an undefined truth value. Their source of truth is their training data. The problem is that human text is not tightly coupled to the capital T truth.
Nor is the LLM output tightly coupled to the training data. They'll "eagerly"[1] fill in the blanks wherever it sounds good.
[1] here I don't mean to imply agency, just vigor.
“Toked out” is really, really good, thank you for this
Several people I know have already gone through phases like this. When you're doing it alone there is a moderating factor when their friends and family start calling them out on their behavior or weird things they say.
I can't imagine how bad it would be if your employer started doing this from the leadership. You'd be pressured to get on board or fear getting fired. Nobody would be trying to moderate your thinking except your coworkers who disagree with it, but those people are going to leave or be fired. If you want to keep your job, you have to play along.
I suspect we're going to see this in many corporate environments soon, if we aren't already
Personally I expect that I will be this person soon, probably fired. I'm not sure what I will do for a career after, but I sure do hate AI companies now for doing this to my career
this is exactly what is happening. instead of building true AI culture around thoughtful adoption of AI strengths while defending against weaknesses, they're coming up with bullshit heuristics like "every repo has a CLAUDE.md", watching private token usage dashboards, and terrorizing everyone into doing it (or lose your job).
this leads to naive AI adoption, which is the worst of both worlds (no real speedup, out sourcing thinking, ai slop PRs, skill rot).
I have a friend that is a junior in a security-oriented sys-admin/network engineer type role. They have been doing the job for only a bit over a year. No background in programming.
Their entire organization has been handed Codex/Claude and told to "go all in on AI" and "automate everything". So the mandate is for people that do not know how to code and have the keys to the castle to unleash these things upon their systems.
This is at a large organization with tens of thousands of employees.
I am waiting with bated breath for the ultimate outcome!
There has never been a better time to be in incident response and adjacent fields.
From what I have seen, most corporate it security people are at a service desk level at best. They are tool runners who don't really understand what the tools spit out, they just go bug other teams about it.
I agree with you, except it isn't even good at writing code. Almost every time that you get an LLM to write a bunch of code for you, it has mistakes in it. The logic isn't right, the API calls aren't right, the syntax isn't right (!). That problem hasn't yet been fixed and it looks as though it never will be. That means that every line of code it generates, you have to review, because even if 95% of the code is correct, you need to find the 5% which isn't. But if you have to do that, it becomes slower than just writing the code yourself. As people have pointed out over and over again: typing in the code was never the part that took time. So I don't agree that LLMs are really useful for writing code.
LLMs are good at producing code that seems plausible at first glance and appears to work, but it never really does. And when trying to fix things, you discover 7 slightly different ad hoc implementations of the same thing, with their own weird edge cases and behaviors. And you likely miss 4 more. There is no intention or coherence behind any of it.
I'm seeing it with lawyers, too. Like, about law. (Just not in their subject matter.) To the point that I had a lawyer using Perplexity to disagree with actual legal advice I got from a subject-matter expert.
when you outsource thinking to AI, you get that magical speed up. the agent is making decisions for you, so things move at agent speed. it often makes decisions without telling you, and the final "here's the plan" output often requires you to understand the problem at great depth, which requires return to human speed, so you skim and just approve.
the trick is to be mindful, aware, and deliberate about what decisions are being outsourced. this requires slowing down, losing that absurd 10x vibe coding gain. in exchange, youre more "in-the-loop" and accumulate less cognitive debt.
find ways to let the agent make the boring decisions, like how to loop over some array, or how to adapt the output of one call into the input of another.
make the real decisions ahead of time. encode them into specs. define boundaries, apis, key data structures. identify systems and responsibilities. explicitly enumerate error handling. set hard constraints around security and PII.
tell the agent to halt on ambiguity.
a good engineer will get a 2x or 3x speedup without the downsides.
Those kind of advice ultimately don't matter. If you're familiar with a programming project, you'll also be familiar with the constructs and API so looping over an array or mapping some data is obvious. Just like you needn't read to a dictionary to write "Thank you", you just write it.
And if you're not, ultimately you need to verify the doc for the contract of some function or the lifecycle of some object to have any guaranty that the software will do what you want to do. And after a few day of doing that, you'll then be familiar with the constructs.
The only way to do that is if you have implemented the algorithm before and now are redoing for some reason (instead of using the previous project). If you compare nice specs like the ietf RFCs and the USB standards and their implementation in OS like FreeBSD, you will see that implementation has often no resemblance to how it's described. The spec is important, but getting a consistent implementation based on it is hard work too.
That consistency is hard to get right without getting involved in the details. Because it's ultimately about fine grained control.
If there's one thing I know about users is that they're never certain about whatever they've produced.
Can concur. I would say I am doing 3 things per day now instead of 1.
I think the author means that we as homosapiens cant stop talking about this new shinny hammer we just invited
I am starting to come around to a similar sentiment. I have seen several large projects cook now for almost a year are not done. These are not trivial projects but the leads are heavily using ai at every opportunity.
I wasnt before but I am 100% confident that AI has done nothing to speed the delivery. It hasnt slowed it down either. It is a wash. The job is more miserable though.
Ai gives generic answer for ideas but it's great for code. Pattern matching works for one not the other.
What I'm seeing is a little eternal September of support tickets about programs that fail to interface the JSON API of a customer of mine. The API is always allucinated. In the best case there are out of place attributes. Often they don't exist at all. I've seen x, y, width, height when we have only top and left. Of course no human read the documentation. Those are probably founders vibe coding a client without the technical competence of understanding the API doc on Postman. That is understandable. Unfortunately they don't even have the competence of pointing their AI to Postman in the right way. My custumer assessed that they will always find a way to do a mistake despite any mitigation from our side. What I do is replying to those tickets with line by line comments of the allucinated JSON. I never talk about AIs because I might hurt the pride of some of them and, who knows, some little mistakes could be from real junior developers. Sometimes the tickets are followed up by more puzzled ones, sometimes they fix the problem. Probably they copy and paste my reply to their bots.
You must not give in to the temptation to mention pirate talk, Klingon, or goblins.
But now that I've put the seed in your mind, you probably (hopefully) will. :)
I've heard the same thing mentioned by a close friend building integrations. They are helping/supporting real use cases but they decided not to help vibe coder founders without an understanding of how APIs work etc. It's just too big of a gap to cover even for larger companies with strong support.
Seeing this too. Customer support tickets are all AI now. The random bolded words, the em dashes, they way where if you KNOW what is actually happening, they are slightly off or just WAY off.
If you were tapped into AI first features you'd design aliases in your api so AI hallucinated api exists for next time
Create a <domain>.tld/llms.txt or some SKILL.md files. When I encounter these tickets I just share links to these resources and the problem goes away.
That's why I made an API LLMs can call to search fastcomments APIs: https://docs.fastcomments.com/guide-llm-kit.html
No it isn't. Do you believe what teachers told you in school? Yes? Well, I guess you're suffering from just normal psychosis!
I don't understand how people don't understand that people offer unreliable information too. We learned about the tongue map in school as kids - many kids still learn that in school today. It's still BS regardless whether it was told to you by a teacher or AI.
You don't suffer from psychosis for believing a source of information, you're simply mistaken. You need a more critical eye to assess what you're told in general, not just AI.
Nope. At least, not without proof. That would, IMO, be kinda crazy. We could argue semantics - maybe “stupid” would be a better word? Lacking in critical thinking skills? Whatever “it” is, it isn’t good.
There's a huge difference between a teacher giving outdated information representing what was once our (or at least their) best understanding of the world, and a chatbot that just randomly makes up things for no reason while insisting that it's all true.
Also, a good teacher should be encouraging the development of critical thinking skills and correcting your errors, while AI will just tell you how brilliant you are when you wrongly tell it about how you've just invented a new form of math or disproved a scientific theory you barely understand in the first place.
Not all BS is the same, just as not all sources are equally unreliable.
If you think you can let AI write code without double checking you have AI psychosis.
If you prefer reviewing AI-written code over writing it yourself, you just have odd preferences from my perspective (but not psychosis).
What does 'prefer' mean here?
I would say writing it myself is more enjoyable (in some cases). But I quite understand that I am not paid to enjoy myself. I'd say it's quicker getting AI to do it and reviewing. I believe the outcome is no worse on average. So yes, that's my chosen approach.
I've been strictly using LLM's to either push stuff that I've done plenty times before and are mostly boilerplate or have zero value for writing them by hand (not even educational), and I always ENSURE that they work on stuff that are easily verifiable and proven incorrect with my existing knowledge or a few minutes of googling.
What is "thinking"?
LLMs can do advanced math and coding, which involves logic, so they are definitely capable of using logic. Which is what most people call reasoning.
So "LLMs are incapable of reasoning, they are just pattern matchers" is wrong. A lot of logic _is_ pattern matching, BTW. Like, syllogisms - deductive reasoning - do you think LLMs are incapable of that?
The thing you're referring to is that LLMs are trained to produce an answer which a human would like, i.e. they aim to produce plausible rather than correct answers.
So it's not so much a mental deficit as a different goal. Trusting LLM blindly is definitely dangerous, but dismissing it as useless for anything by code is rather wrong.
Pattern matching is hardly what distinguishes human from LLM - if you ask somebody a question about policy, for examples, chances are they'd just recite something they heard somewhere, never really thinking about it from first principles.
I’ve been talking to a lot of engineers about how they use AI in their day to day and it’s dramatically different than what you see from the hypers.
The vast majority use one agent at a time and careful step through code. The main benefit they report is often about researching the codebase and possible solutions.
Aren't we ignoring the elephant in the room ?
Garry Tan has been the primary crusader for AI driven decision making. I'm sure his position is more nuanced, but his twitter driven communication makes him appear like a caricature of a man in AI psychosis.
When the head of YC champions AI driven decision making, companies will inevitably be influenced into doing exactly that. It's unfortunate, because AI is generational technology and the hyperbole distracts from the real sea change occuring in labor markets everywhere.
Today's frontier models are genuinely useful as rubber ducks or grunt units. They are horrible for actual problem solving. These tools are not capable of actual reasoning. They will happily crap out a broken, untyped, untested Next.js monstrosity with no discernible architecture. They will build esoteric shell scripts to perform operations that could be done idiomatically and simply with tools already in your codebase. They will tell you to walk to the car wash then have the car wash valet your car back to you when confronted with the flaw in their logic. They will validate incorrect beliefs like ketchup being an acceptable hot dog condiment or the notion that "The Red Hot Chili Peppers" make good music. They have no taste, no anima, no drive.
Rule #1: Do not anthropomorphize the LLM. It is a million monkeys at a million typewriters piped into a digital sieve. I don't know how or why people place such trust in them while bemoaning other technology in our lives for being so broken ("my algorithm [sic] only shows me X", "the new iPhone update sucks", etc). If everybody followed this rule then the deluge of emoji-ridden hokum pouring into Slack workspaces and GitHub PRs around the world would cease but I'm not holding my breath.
Part of the psychosis are AI usage mandates, where companies require a certain amount of LLM usage per worker. Of course these things are useful, but forcing them on workers is psychotic.
Mitchellh is on to something. Some of the AI products I've seen seem like psychosis hallucinatory fever dreams, using terms and concepts that have no meaning. Funding? $50,000,000 pre-seed.
When I note some strange money flow I just assume embazzlement or money laundering.
This is a critical communications issue that is becoming what I believe the defining characteristic of "This Age": nobody knows how to discuss disagreement, and because it cannot even be discussed communication ends, followed by blind obedience, forced bullying, retreat and abandonment. This is going to be a hell of a ride, because nobody can really discuss the situation with a rational tone.
while I find myself agreeing with the posted tweet, I don't disagree that we need to communicate about things we disagree about better. Take my upvote
We're definitely in the mess around phase of AI adoption.
I don't think it's super clear what we'll find out.
We've all built the moat of our careers out of our expertise.
It is also very possible that expertise will be rendered significantly less valuable as the models improve.
Nobody ever cared what the code looked like. They only ever cared if it solved their problem and it was bug free. Maybe everything falls apart, or maybe AI agents ship code that's good enough.
Given the state of the industry were clearly going to find out one way or the other, hah!
I think some companies will find out that their senior engineers were providing more value and software stability than they gave them credit for!
Corporate feedback loops are very slow though, partly because management don't like to admit mistakes, and partly because of false success reporting up the chain. I'd not be surprised if it takes 5 years or more before there is any recognition of harm being done by AI, and quiet reversion to practices that worked better.
The AI psychosis is not the anti-opinion to the use of AI.
I use AI coding tools every day, but AI tools have no concept of the future.
The selfish thinking that an engineer has when they think "If this breaks in prod, I won't be able to fix it. And they'll page me at 3AM" we've relied on to build stable systems.
The general laziness of looking for a perfect library on CPAN so that I don't have to do this work (often taking longer to not find a library than writing it by hand).
Have written thousands of lines of code with AI tool which ended up in prod and mostly it feels natural, because since 2017 I've been telling people to write code instead of typing it all on my own & setting up pitfalls to catch bad code in testing.
But one thing it doesn't do is "write less code"[1].
[1] - https://xcancel.com/t3rmin4t0r/status/2019277780517781522/
Maybe it's just my prompt or something but my coding agent (Opus 4.7 based) says things like "this is the kind of thing that will blow up at 2am six months from now" all the time.
It's really inconsistent though.. it takes shortcuts and leaves todos all the time without really calling it out explicitly, you have to pay close attention.
Why do you all still submit twitter.com links when that domain does not even work?
HN needs to start sticking an ?mx=1 on the end
https://news.ycombinator.com/item?id=41173763
I don't doubt there are companies totally misusing coding agents and LLMs in production. There are also real companies with real revenue and solid architecture using LLMs to deliver products. There are also companies with real revenue and rapidly accumulating tech debt.
Eventually the companies that can't cope with undisciplined engineering will succumb to unacceptable reliability and be outcompeted, just like in the "move fast and break things" era.
"Just use autoresearch and it will fix your app's memory leaks in an hour" is what I was nonchalantly told by someone who has never written a line of code ever.
I guess what I relate to the most is how dismissive people get about real software engineering work.
I may have skill issues, but I am yet to reach the level of autonomous engineering people tend to expect out of AI these days.
I'm going through a mixed experience regarding this, personally.
Management is really pushing AI. It's obnoxious, and their idea on how it fits into my team's job specifically is completely, hilariously detached from reality. On the off chance someone says something reasonable, unless it fits the mold, it's immediately discarded. The mold being "spec driven development". We're not even a product team for crying out loud. I straight up started skipping these meetings for the sake of my sanity. It's mindwash, and it's genuinely dizzying. The other reason I stopped attending is because it ironically makes me more disinterested in AI, which I consider to be against my personal interests on the long run overall.
On the flipside, I love using Claude (in moderation). It keeps pulling off several very nice things, some of which Mitchell touched on in this post (the last one):
- I write scripts and automation from time to time; Claude fleshes them out way better with way more safety features, feature flags, and logging than I'd otherwise have capacity to spend time on
- Claude catches missed refactors and preexisting defects, and does a generally solid pass checking for defects as a whole
- Claude routinely helps with doing things I'd basically never be able to justify spending time on. Yesterday, I one-shotted an entire utility application with a GUI to boot, and it worked first try; I was beyond impressed.
- Claude helped me and a colleague do some partisan cross-team investigation in secret. We're migrating <thing> and we were evaluating <differences>. There was a lot of them. Management was in a limbo, unsure what to do, flip-flopping between bad options. In a desperate moment, I figured, hey, we kinda have a thing now for investigating an inhuman amount of stuff in detail - so I've put together a care package for my colleague with all our code, a bunch of context, a capture of all the input data for the past one week, and all the logs generated. Colleague put his team's side of the story next to it, and with the help of Claude, did some extremely nice cross-functional investigation. Over the course of a few weeks, he was able to confirm like a dozen showstopper bugs, many of which would have been absolutely fiendish if not impossible to fix (or even catch) if we went live without knowing about them. One even culminated in a whole-ass solution re-architecturing. We essentially tore down a silo wall with Claude's help in doing this.
So ultimately, it really is a mixed bag, with some really deep lowpoints and some really nice higlights. I also just generally find it weird that a technical tool [category] is being pushed down people's throats with a technical reasoning, but by management. One would think this goes bottom up, or is at least a lot more exploratory. The frenzy is real.
What's the matter with spec driven development? It probably carries derisk IP benefits
This will be pushed down from people, who will have no deep understanding of it. But it does check some boxes in an ISO certification.
Well, now you must to work with a confusing tool which slows you down. You are not allowed to use claude directly anymore, because someone heard that mythos is really bad for security. But hey, the tool integrates well with Jira!
You hate every second working with this thing. All the joy you had with explorative coding is forever gone, which was the sole reason you entered this field.
Deep inside you know that you can't change your job, because every other employer will cut its workforce as AI removes all manual labor of a software engineer and reduces risk to a minimum.
Oh, now we can finally move all those jobs to india without risk and shareholders will love it! How awesome is that! Wait, do we still need that guy in cubicle 42, who bitches and moans about AI every day? Nah...
Totally agree with one shotting GUI tools. I especially have liked it to create a single-file web app, and then open it with Chromium locally (no web server needed).
In my case, it built a tool for splitting sounds and a tool for defining hitboxes for a game. Tools made exactly for more workflow. Wild times.
Welcome to the club, Mitchell! Pizza's to the right.
In all seriousness...well, yeah. AI is a monkey's paw, and that's how monkey paws work. So many movies and books warned us!
You just have to wish for the rest of the monkey.
Sounds pretty accurate. Bunch of comments on this thread sound like AI is some kind of a new doomsday cult. The most annoying thing I find personally is that all engineering principles are getting crushed by non techies. Management counting token usage, forcing agent use, reducing headcount in the name of productivity gain. Devs building bridges but nobody knows what the bridge is, what are the standards to which it was built, how it works and how to maintain it. VCs counting extra money claiming chasing the holy profit is the future. The abundance of engineering apathy is disturbing.
I think AI rescue consulting is going to be come a significant mode of high value consulting, similar to specialists who come in to try and deal with a security breach or do data recovery.
Purely AI written systems will scale to a point of complexity that no human can ever understand and the defect close rate will taper down and the token burn per defect rate scale up and eventually AI changes will cause on average more defects than they close and the whole system will be unstable. It will become a special kind of process to clean room out such a mess and rebuild it fresh (probably still with AI) after distilling out core design principles to avoid catastrophic breakdown.
Somewhere in the future, the new software engineering will be primarily about principles to avoid this in the first, place but it will take us 20 years to learn them, just like original software eng took a lot longer than expected to reach a stable set of design principles (and people still argue about them!).
AI janitors
It's kind of like producing code is becoming more like farming.
We didn't create the dna we rely on to produce food and lumber, we just set up the conditions and hope the process produces something we want instead of deleting all the bannannas.
Farming is a fine an honorable and valuable function for society, but I have no interest in being a farmer. I build things, I don't plant seeds and pray to the gods and hope they grow into something I want.
Prayers are for weather. Pretty much all farmed plant, animal, and fungus species have been selectively bred or genetically modified. Farmers know what's going to grow.
Farming has merely a lot of study and input into the process, very little actual control and no determinism at all. We know how to improve chances is all. The fact that we breed and "engineer" is like a drop in the bucket.
It's pretty deterministic in that if you plant corn you will grow corn not beets, you know?
If the farming situation were as dire as you seem to suggest, we'd have unpredictable famines all the time, but we don't
You might grow corn, or you might grow defective unusable corn and/or any number of other things like locusts or fungi or other plants that decide to grow in the place where you planted corn. Sure, the corn seeds will not produce ball bearings. Genius observation. There are about an infinity of other things that can and do happen besides that.
Planting is merely setting up the conditions. We didn't write the dna, we couldn't write the dna if we wanted to because we are an infinity away from understanding all the actual processes that descend from the dna. And when we utilize the dna that we simply found and didn't and couln't hope to write, it's always, at best, a case of hoping it goes right again this time.
Tell me you've never done any farming without telling me you've never done any farming. There is certainly risk in the business due to market fluctuations, weather, natural disasters, disease, and pests. But the final product is highly deterministic. Almost all genetic variability has been expunged from major food production species in a relentless pursuit of predictable yield. Everything looks and tastes the same. We can debate whether that's a good thing but it is the reality for most farmers.
If it was deterministic, there would be no such thing as blights and other forms of failures. There would be no problem with the bannannas, or coffee or wine grapes. There would be no such thing as a critical few days of the entire year where if anything goes wrong you lose the entire year because it was too humid or too cold or your equipment was out of commission for a week. The bees wouldn't matter at all.
Even when it works, even if you put in a lot of work and experience and understanding, it still just worked by itself and it's just good luck every time.
You have also guessed incorrectly.
Not janitors. Hazmat cleanup crews.
Like this: https://en.wikipedia.org/wiki/Times_Beach%2C_Missouri
Scrape off all the soil, put it in casks, and bury it in a concrete bunker for 10000 years. Then relocate everyone and attempt to rebuild.
Wow, it’s true, AI really is set to match human performance on large, complex software systems! ;)
The origin of 'dark DNA' begins to make more sense through this sort of lens, except the system somehow maintained a level of compensation to fix all its flaws.
We do as well, it's called bankruptcy. Not every company survives but in the end the ones that do are more resilient.
Humans who have been writing systems like that for many years know how to maintain and modify them successfully. It’s just that our industry has a bias towards youth who don’t think they have anything to learn from those who came before them.
How do you explain to a junior this pile of messy code isn’t crap but is actually years of integrated knowledge ? That the most common principles discussed in computer science (OOP, SOLID, DRY etc.) are actually just little guides that aren’t to be taken to the extremes ?
tell them they need to turn a profit as quickly as possible
Wait if they can do that they’re not juniors anymore :P
Expert knows when to break the rules
Experts take the time to learn why the fence was there in the first place.
Experts have beginner’s mind.
Experts are people who have made all the mistakes there are to make in their chosen field.
Including all of the above.
It's a dice roll to keep the junior around until he unlearns the wrong bits.
Here's a 26-year old post on the exact topic of messiness you raise:
https://www.joelonsoftware.com/2000/04/06/things-you-should-...
A decade ago, I was sitting in on a meeting about a rewrite and, before I could say anything, someone in the first year of her career asked why anyone thought a rewrite would be any cleaner once all the edge cases were handled. Afterwards, I asked her where she learned this. She said "I don't know, it just seems kind of obvious." She went on to be a great engineer and is now a great manager.
The bolded quote "It’s harder to read code than to write it." is hilarious given todays context... it has only become more true :)
I work on internal facing software and every rewrite I've seen in 20 years suffers from the same symptoms. The code/system is a mess because it has been exposed to reality for a decade. Reality is messy. That's why they pay us money, believe it or not.
Greenfield guy comes in, promises the world, and starts from some first principles white papered architecture. It's really lovely until they onboard the first user. Then they slowly commit all the "sins" (features that drive revenue) of the first system.
The firm is stuck supporting N systems indefinitely because the perfect new system takes so long to cover even 30% of the original system use cases, that management takes a flier on.. bear with me.. a second rewrite. Now they have 3 systems.
I've seen more 3rd systems than I've seen actual decommissioning of original systems into a single clean new system.
The answer is chipping away, modularizing, and replacing piecemeal Ship of Theseus style. But that does not drive big hires and big promotions.
Do they??
I believe this type of person exists.
My team lead has worked on the same software for 30 years. He has the ability to hear me discuss a bug I noticed, and then pinpoint not only the likely culprit, but the exact function that's causing it.
I do the same thing in a project I’ve worked on for 25 years. I’ve had mediocre at best results with AI. It’s useful to discuss concepts with, but the code never handles the nuances of the edge cases.
Then they quit or die.
Yep this is like comparing master craftsmanship with a production line. You're gonna get good attention to detail and a masterpiece from one, and a limited thing that will break after few years from the other. But for majority of use cases the second one is enough. And pointing out the master craftsmanship is "better" is besides the point.
And with one you need to train a guy for 25 years and with the other you need plan mode for a few minutes and then it runs 24/7.
Our society needs more experts, not less.
What is your argument? Should we stop training people on how to do something because we're mortals?
Yeah... in my experience people who code like that 'successfully' make modifications that fix an immediate problem while kicking another bug or two further down the road in a never-ending sunk-cost-fallacy of job security...
Yes.
There is a lot of absurdly complex software that runs with high reliability. We hear a lot about the ones that don’t.
Executive leadership bias older not younger, no?
No.
Younger implies cheaper.
This is sadly so true.
I have really tried as an "old" person in the field to try and pass on the stuff I've learned, but "craft" and such really has absolutely no home in modern dev culture. The people who care about history, the craft, etc. are increasingly rare.
it's been 10y and i still haven't seen a human system that bad
maybe some that people said were that bad. but they just needed some elbow grease. remember, it takes guts to be amazing!
Reminds me of the quote in the original Westworld movie:
“ These are highly complicated pieces of equipment… almost as complicated as living organisms.
In some cases, they’ve been designed by other computers.
We don’t know exactly how they work.”
Now how did that work out ;-)
However Michael Crichton imagined it would.
I guess that “well” wouldn’t have sold many books.
That depends. If this equipment is part of the plot, you're right. If it's part of the premise of the world, "well" would be the expectation.
Shelve it with the Jurassic Park version where John Hammond builds a safe, profitable theme park, and The Andromeda Strain that gives people the sniffles.
As the models keep improving, wouldn’t you be able to task a newer AI to “clean up this mess”?
How could anyone answer that with any level of certainty?
Frankly this is what everyone is counting on whether they know it or not. The question though is not “will the models get good enough?”. The question is does the repo even contain enough accurate information content to determine what the system is even supposed to be doing.
Ai runs `rm -rf`
Beyond the Singularity, we reach the Nullarity.
https://youtu.be/m0b_D2JgZgY
Are they improving? I thought they were just getting more expensive
Mythos apparently wrote a poem so beautiful it made Dario cry.
Crocodile tears, just like the fake "fear" of its capabilities. Anything to raise another round of dumb oil money.
Roses are red
Violets are blue
AI is great
And so are you
Every frontier model from each major US lab is cheaper than their frontier model this time a year ago with the exception of Anthropic whose pricing has remained exactly the same.
People are often skeptical when I say this, but there's simply no guarantee that it's possible in principle to clean up a bad architecture. If your system is "overfitted" to 10,000 requirements from 1,000 customers, it may be impossible to satisfy requirements 10,001 through 10,100 without starting over from scratch.
It may be difficult, but impossible is such a big word to use here
It's really not that big of a word. The CAP theorem shows that as few as three reasonable-sounding requirements with no obvious conflicts can be impossible to satisfy simultaneously. (User needs will start more flexible than strict mathematical requirements, of course, but once people start to build production workloads on top of your systems that flexibility is radically reduced.)
How is a newer AI going to "clean up" dropped databases, compromised computers or leaked personal data?
(None of above is theoretical)
I really am surprised that people on a heavy CS themed forum still have trouble grasping this.
Imagine the year is 1995, C exists, but some guy out there is working on essentially what modern Python is. He says to you "check out this language, you can just import stuff, and use it and dynamically modify anything at run time". You can probably come up with hundreds of arguments about things that could go wrong, like memory clean up, threading, e.t.c, but turns out, incrementally, they were all solved and we have the modern Python that basically is good enough to build these large LLM models.
Now imagine modern programming and computing is what C was back in 1995, and AI use is that guy building the Python code.
What's your point?
I think you have some serious misunderstanding here.
The point is that in the future, AI will be able handle things like missing databases just like the modern high level dynamic languages can import a library to handle whatever you want.
I can't tell if you're being facetious, but a future AI really may be able to fill in a missing database. Like, if it knew some of the entries, it could infer the rest.
Wow - imagine being able to infill a geophysical database with the dullest possible milquetoast totally expected signal derived from the NASVD most common eigen vectors.
The infill will look seamless.
And entirely lack any actual strikes of interest - the outliers are exceptional signal and the entire raison d'etre for building such a database.
Jeez, if AI can just infill where the gold is, why even bother to look in the first place.
Thats not what I mean.
The original question was
For each of those things, you can right now build an agent that handles all of that. Or use a large frontier model with enough context to build code that ensures all of those edge cases are handled.
Future coding will essentially be like this. The concepts of dynamic vs compiled language will shift towards having frontier edge models put together code versus small runtime edge models dynamically processing data.
You can imagine anything you want, but it’s not an argument - you could apply this to anything. “Python was successful after a dubious beginning so NFTs will be successful”
Also, Python does not build or run large language models. It orchestrates C code that does that, and it was probably good enough to do that in 1998.
Highly dynamic languages existed for decades prior to 1995, Python was not particularly innovative in its features at the time. There were also countless languages more feature-rich than C being used for development at the time.
The biggest change that happened was that hardware kept getting better and it became feasible to use garbage-collected languages everywhere including really inefficient implementations like CPython.
That being said, 30 years later Python is still slow as shit even compared to other dynamic languages and runs into all kinds of scaling issues when used for anything serious. And everywhere that performance matters, software continues to be written in typed, compiled languages including C (but also C++, Rust, Go, etc.). Even in ML, Python chiefly acts as a thin wrapper and glue language for high performance CUDA libraries (aka C and C++).
So your historical analogy is mostly anachronistic.
No, you just don't have a grasp on reality. For example, you claim that Python runs into scaling issues for anything serious, but you are blissfully unaware that youtube and uber both run python backends. Nobody cares that its "slow" by whatever metric you consider. Its fast enough. The metric that matters is developer time not compute time, because the former is vastly more expensive. Python and Node are the number one languages on github for a reason. And you are vastly deluded on how many jobs there for C++ and Rust devs lol.
In the future, you won't be dealing with strings, json, or apis. You will be importing agents, and giving them brief instructions, either in plain English or in some intermediate language higher than Python that is more brief. Wanna deal with database reliability ? Import database agent and give it brief instructions on what you want to manage. Just like you mention, right now Python is the wrapper for low level libraries, because everyone who is doing work in ML doesn't want to waste time making sure their C Cuda kernels compile. In the same way, nobody is going to care if they get the API headers right, or if their strings are correctly parsed when you can just invoke a dedicated LLM (which will likely be highly specialized small model able to run on local hardware) to do all that.
You can scream and cry as much as you want how that is bad, how its slow, but nobody is going to care because shit is going to get built faster. Ever notice how despite the massive layoffs across tech, there isn't service degradation in any sector? Good luck trying to sell your Rust skills in the future lol.
Someone responded to a previous comment of mine [0] positing a Peter principle [1] of slopcoding — it will always be easier to tack on a new feature than to understand a whole system and clean it up. The equilibrium will remain at the point of near, but not total, codebase incomprehensibility.
[0] https://news.ycombinator.com/item?id=48037128#48038639
[1] https://en.wikipedia.org/wiki/Peter_principle
Yes. And as the models get better, it works better. But at one point you do have to understand the code because it's also just guessing as to what your actual intentions are.
It doesn't know what mess you want to clean up. A lot of times AI just starts making up new patterns on top of other patterns and having backwards compatibility between the two. How does it know which one you actually like?
A non-technical friend of mine has just won some hospital contracts after vibecoding w/ Claude an inventory management solution for them. They gave him access to IT dept servers and he called me extremely lost on how to deploy (cant connect Claude to them) and also frustrated because the app has some sort of interesting data/state issues.
What concerns me about this is that as these stories multiply and circulate people will just completely stop buying software/SAAS from startups, because 90% or more will be this same thing. It will completely kill the market.
Or you end up with a certification process, which will of course introduce it's own problems but startups doing things the right way and not just "moveing fast and breaking things" can thrive.
Oracle have routinely had multimillion pound contract failures and people keep buying from them. Big vendors are too big to fail.
Same with Deloitte
no one's getting fired for hiring either one.
Those are custom software or heavily customized implementations of ERP and similar systems for very large organizations. I’m talking more about the SMB market where today it’s possible for a small team to carve out a niche and make a nice living or even bootstrap a venture that competes with a large player that has poor UX or antiquated feature designs.
The reason Oracle can continue failing at those massive projects is simple: everyone fails at them routinely and often it’s the customers fault.
I used to gripe about various ERP companies but after having dealt with enough, yeah, that's just what the world of ERP systems is like. You will spend your time even with the best of them desiring to scream endlessly at everyone who works there. And they also know your pain but are powerless to help.
there are no 2 identical deployed ERP systems.
It's just an umbrella term for "weak process glue code".
It's even simpler. Youre not paying oracle for some delapidated HR system. You're paying for the legion of accountability that is their on-site engineers to fix stuff for you when things screw up. You're essentially subscribing to a team of engineers you don't need to directly pay salary and benefits to.
People who think you can out efficiency that kind of accountability don't understand how large orgs think.
it will kill all the people in that hospital too
What is this, Humanitarian News?
thankyou
The real Hackers were the ones actually trying to minimize suffering all along. Not reproduce it at scale.
But the Torment Nexus is such an interesting technical challenge! and I don’t personally torment people: I just move protobufs around! - Software Engineer #1 and #2 excuses
Yeah but only one of those actually puts those responsible in prison https://en.wikipedia.org/wiki/Elizabeth_Holmes
Those patients weren't hurt. Totally different from the post you're replying to.
I mean, the stories about how stuff was getting built in the late 90s/early 2000s aren’t much worse.
jfc lmao
This hospital will learn some hard lessons. I hope their backup strategy is good. I'm surprised they can field software from an entity that isn't SOC2 & HIPAA certified.
No worries! At worst, the contractor can just tell Claude to make sure the hospital knows they're appropriately certified. And the hospital can use Claude to make sure the certs are valid. Everybody wins, except the ones who end up dead. Or with their health destroyed.
What do you think the fake Delve attestation scandal was about? https://news.ycombinator.com/item?id=47444319
Don't help him. Let him figure it out by himself, else they (he and hospital) will never learn.
I don't have time for that. I just told him he needs to hire somebody
I was going to say to open yourself up as a contractor and scape some of the money off top. But it sounds like you dint need that opportunity.
That sadly does seem to be the trajectory of 5-10 years from now, though. I can't speak to if "AI is the future" of 30+ years from now, but these coming years sounds rife for "janitors" to clean up all the slop being produced by newly empowered idea guys
Or, "help" by asking questions, or otherwise by sharing an AI review/analysis/suggestions, since they're into that kind of thing.
Definitely cleaning up other people's AI mess for them for free is not a good use of time.
A hospital could not learn a bigger lesson from this person than their existing big players.
(Screams in "deployed in 2026 a new product that only works in internet explorer" in healthcare).
I work at a university and we still have some workstations that need IE as well, for a healthcare vendor app that needs ActiveX. Up until recently we even had some machines running Windows 7.
This is going to happen all over. Company I'm currently contracting with has gone AI everything (aka technical debt hell), and they're gonna suffer for it. I'm glad my consulting contract ends in 2 months. I don't want to be around for the crash
Have you tried to talk him out of it, and have you considered blowing the whistle on him? He could kill people!
I hope you have quoted him a very very high hourly rate.
Wow. This is like every other gold rush. Millions will walk into the ice and snow, somehow not questioning that their ability to dig is not unique.
Well, selling shovels has always been a good way to deal with that problem
The shovel sellers are ringing the cash register.
As a cybersecurity IR professional as much as I hate to see this happen to a hospital this kind of thing is responsible for essentially tripling my income over the last 3 years.
Did he lie about HIPAA compliance?
Heaven help us.
As a SWE that has only ever worked for an employer or on his own projects, this makes me wonder: how would someone even get such a contract? Did this person already have a consulting business? Do you just call up random hospitals and ask if you can demo an inventory management system for them? Did this person already know people at the hospital? I know technical folks that do independent consulting, but even with a vibecoded product, how is it that anyone can just get such a contract?
Frictional money.
People really have a misconception about the sums of money that companies operate on on a regular basis. If you are a people person and know essentially how to sell yourself, you can "scrape" money on the fact that nobody is going to look or think too hard about some contract that represents a tiny fraction of the years budget.
That still leaves the question of how one gets their foot in the door. Lots of us are aware of the budgets but we don't get how's sales work at that level.
The only way something like this would work is through "networking", and trust that you are capable of delivering.
I'm practical terms, go to where the decision makers are and shmooz with them. It's a numbers game. Eventually someone will say yes.
That's what it means to be a "people person" in the context of trying to sell a product, yes. Getting within 2 degrees of a decision maker can open up millions for you, while being a rounding error for every company you work with.
He's already in the whole consulting sphere around these hospitals in his area.
I'd really like to know how he won contracts, just in general. Did he have some connections. And he doesn't even know how to get it to run on a server by himself? There's millions of people that can do that, if he can win contracts why worry about vibe coding at all, just hire someone to do it. Winning contracts is the challenge in my view.
He's already within the consulting sphere around hospitals in his area.
Hospitals? Vibe code?
Dear Lord. Respect to your friend for mad marketing skills, however. Selling slop to mission-critical sectors is next level.
It's really nowhere near as complicated as making distributed systems reliable. It's really quite simple: read a fucking book.
Well, actually read a lot of books. And write a lot of software. And read a lot of software. And do your goddamn job, engineer. Be honest about what you know, what you know you don't know, and what you urgently need to find out next.
There is no magic. Hard work is hard. If you don't like it get the fuck out of this profession and find a different one to ruin.
We all need to get a hell of a lot more hostile and unwelcoming towards these lazy assholes.
Financial auditing with pre-AI technical chops will be uniquely niche-valuable, too :)
This might not pan out to be the glorious victory of human craft as you’re imagining it to be.
Here’s a slightly different future - these AI rescue consultants are bots too, just trained for this purpose.
Plausible?
I have already experienced claude 4.7 handle pretty complex refactors without issues. Scale and correctness aren’t even 1% of the issue it was last year. You just have to get the high level design right, or explicitly ask it critique your design before building it.
Do you think people are not giving their agents specs and asking for input?
Very often, no.
The ones who end up with messes, no
Maybe the professional devs, but not the vibecoders
A thing I've noticed is that everyone thinks they prompt better than the next guy.
This. I have this buddy, who is not an idiot by stretch of the imagination and more adventurous than me in some ways ( I don't really run agents on my machine ), but when I was looking at his prompts, I sometimes question how he gets anything done at all. It is vague and angry demands.
Not sure about the angry part, but vague sometimes works really good. The important part is to have enough good context pushed into the context window beforehand (codebase explorations, docs, etc). Then a vague prompt of the general direction gives the autocomplete more “freedom” to figure out the “best” approach given the context.
Doesn’t work well ofc in a one shot situation with no context.
Yeah, sorry, I myself was being vague, because I don't want to give any identifying info even by mistake. You are right; generalizations here are not as useful. I was talking about something the lines of 'can you make it better', but without llm having the context to understand what better could potentially mean. For brainstorming sessions, I love to start broad. Admittedly, I have limited experience with agents ( though current project intends to bridge that gap ) so it is possible I am missing something ( plus, to your point, I don't know his full setup ).
I think that will happen. I think several things can be true at the same time:
- AI Hype
- AI Psychosis
- AI keeps getting better and better until it can work around big AI slop code bases
The belief in this is a form of AI psychosis, I think.
Maybe in the future but certainly no evidence of this anytime soon
Here's some anecdotal evidence from me - I cleaned up multiple GPT 4.x era vibecoded projects recently with the latest claude model and integrated one of those into a fairly large open source codebase.
This is something AI completely failed at last year.
Maybe you should try something like this or listen to success stories before claiming 'certainly no evidence' in future?
No evidence? Chatgpt came out 3 years ago. You basically just need to stick a ruler up on a curve
I'm no expert, but the skeptic's opinion I've heard would be to ask:
What evidence is there that we're not at or close to a plateau of what LLMs are capable of? How do you know the growth rate from 2023 to present will continue into 2029? eg. Is it more training data? More GPUs? What if we're kind of reaching the limits of those things already?
Since we're not experts, we treat it as a black box. What are the results? Is the quality of the results improving? Is the improvement accelerating or decelerating?
And the answer appears to be that the improvement is accelerating. So how could it be stopping?
https://metr.org/time-horizons/
I don’t think improvement is accelerating. We went from “computers can’t do these things at all” to “now they can” in a few years with the discovery of transformers, and now we get “it can do the same things, except incrementally better, at a drastically higher cost” every few months.
I don’t think that the current AI paradigm has infinite headroom for improvement, similar to how every other AI approach before it eventually hit a limit.
Ultimately, you are describing a fundamental problem with induction -- Hume's problem of induction to be specific. How can we know that anything that has been shown empirically in the past will continue to be true - we can't. Best to investigate mechanistically:
I don't see why we would assume that we are at a plateau for RL. In many other settings, Go for instance, RL continues to scale until you reach compute limits. Some things are more easily RL'd than others, but ultimately this largely unlocks data. We are not yet compute/energy/physical world constrained. I think you would start observing clear changes in the world around you before that becomes a true bottleneck. Regardless, currently the vast majority of compute is used for inference not training so the compute overhang is large.
Assuming that we plateau at {insert current moment} seems wishful and I've already had this conversation any number of times on this exact forum at every level of capability [3.5, 4, o1, o3, 4.6/5.5, mythos] from Nov 2022 onwards.
I'm more curious about how much more capability they can get before the economy collapses.
I think we're close to the plateau of what LLMs can do, but they will keep improving. IMHO the results are already showing diminishing returns.
The (leading) LLMs work by consensus, like Wikipedia, Openstreetmap, web search engine or opensource movement.
What I mean is if I ask LLM "create a linked list", its understanding (of what I want) is already close to the expected ideal. Just like Wikipedia article on linked list, for example.
But the LLMs will continue to improve in breath and depth of understanding the world, although technically (what they CAN do) they probably already peaked. Similarly, OSS movement technically peaked in the 90s with the creation of compiler, operating system and a database; doesn't mean that new opensource isn't being created.
There is so much money at stake, and so much money pouring into AI development, that I think we are going to continue to see gains for a while. People keep coming up with new agent harness techniques like chain of thought, tool calling, and memories. And then the big LLM companies figure out how to actually train their models to optimize the use of those techniques. To claim that we are reaching the top of the plateau is to claim that we are out of effective ideas for improvement. I think that's a ridiculous claim, the technology is too new. And because of the strong incentives to keep making these things better, it's pretty much a given that people will continue to explore ideas until we really are out of effective ideas. I don't think anyone apart from professional AI researchers have any idea where this is all going to settle.
I have personally had success telling Claude that some AI-written system is too complicated and ask it to rewrite it in a more logical way. This sometimes results in thousands of lines of code being deleted. I give an instruction like that if I see certain red flags, eg:
1) same business logic implemented in two different places, with extra code to sync between them
2) fixing apparently simple bugs results in lots of new code being written
It’s a sign I need to at least temporarily dedicate more effort to overseeing work in that area.
I somewhat agree with the AI psychosis framing of the OP. It takes some taste and discipline to avoid letting things dissolve into complete slop.
It's amusing to me that:
* A belief that AI will keep getting better, presented without evidence, does not yield a lot of skepticism around these parts.
* Your comment saying it is wrong to believe AI will keep getting better, also presented without evidence, is downvoted.
AI is currently, actively getting better. That might stop in a year, and you can argue whether it’s linear or exponential, but it’s very difficult to argue that it won’t get better in the near term. On the other hand, arguing that we hit a plateau at this time is just ignoring reality.
This doesn't address anything I said in the above comment.
There are untold billions of dollars to be had if you can make this future come to pass. You don't need AGI to make it happen either. You just need to keep making the context windows bigger and keep coming up with updated training data. It's not the outcome I want, but it really does feel within reach. The only limiting factor is going to be token count and cost to process/generate those tokens. But if you don't particularly care about quality, costs are going to have to go up by several orders of magnitude before you start to regret firing your software engineers.
I don't know what happens in a decade when there are no junior engineers, skilled senior engineers are becoming rare, and the only data left the train LLMs on is 200th-generation slop. But AI slop being qualitatively slop is not enough of a obstacle to prevent that future from coming to pass. And billions of dollars will be "saved" along the way.
Companies are already putting billions out there just to secure and produce training data. And that's the isseue; spending X billions to make X-Y) billions isn't a profit, it's a gamble hoping Y becomes negative (or at least close to zero with a commodity that is profitable) . Real profits have not been made directly from the work on AI as of now. It's made from marketing a narrative of AI working.
That's what makes this whole house of cards dangerous. The prescription to psychosis is profitable. Aka, selling a grift.
With GPT 5.4 or 5.5 I did not notice degradation in performance when it was working on a large 5k line file containing a WebView, JS scripts, as well as native UI.
I instructed it to split it up anyway, yet I wonder how often the concerns around the mess are imaginative rather than practical.
One AI can't vibe code out of the mess, so you'd make another AI trained on getting out of vibe coded messes?
That's serious levels of circular thinking right there.
This is literally how training humans have worked for thousands of years.
We train humans to do things untrained humans can not do.
No it's not, don't be facetious.
That's not at all how AI training works.
Humans, unlike LLMs, are capable of reasoning and thinking. Thus humans, unlike LLMs, can actually be taught and improve.
And the bots training the bots are just bots that were trained to train bots?
Nothing that sexy, just thirty odd years of software engineering data from humans.
Commits, design reviews, whitepapers, code reviews, test suites. And pretty concerning : chat logs and even keystrokes from employees nowadays.
The way we train specialized bots now is incredibly inefficient, that part is rapidly improving.
I thought the same when I saw development outsourced to Indians that struggled to write a for loop.
I was wrong.
It turns out that customers will keep doubling down on mistakes until they’re out of funds, and then they’ll hire the cheapest consultants they can find to fix the mess with whatever spare change they can find under the couch cushions.
Source: being called in with a one week time budget to fix a mess built up over years and millions of dollars.
What happened after development was out sourced to Indians: developer salaries continued to rise much faster than general wages.
If you work like you're outsourcing to the worst consultancy firms, your use of AI will be ... pretty productive, actually.
The complexity you would come to the rescue to solve, would that be from AI or from the style of programming you let the AI have? I mean, you have very different problems if you use functional style vs object-oriented. It is up to the programmer to realize they want a functional style and request that from the AI, as much as possible. Even AI cannot imagine every state transition, unless it is so smart that it should be the one telling you what to do.
Heh. Got a customer recently around this. Entire infrastructure and CI/CD vibecoded. They half implemented Kubernetes in Github Actions that were several thousand lines long and impossible to understand.
I think the problem will get worst. I dislike the marketing around AI, but I do think it is a useful tool to help those who have experience move faster. If you are not an expert, AI seems to create a complex solution to whatever it is you were trying to do.
I've been watching non-developers vibe code stuff, and the general failure mode seems to be ignorance of 3-pick-2 tradeoffs.
They'll spam "make it more reliable" or some such, and AI will best-effort add more intermediary redis caches or similar patterns.
But because the vibe coders don't actually know what a redis cache is or how it works, they'll never make the architectural trade-offs to truly fix things.
I’ve noticed something similar with vibecoded game rendering logic submitted by peers. Sometimes it will be peppered with extraneous checks for nullptr, or early returns on textures that have zero size.
I often wonder if it’s the statistical nature of the LLM mixed with a request in the prompt.
AI LOVES defensive coding. I asked you for code to filter and reduce an array. I didn't ask you for a method that makes sure the array exists and is an array before it does anything else.
But it's so easy now to redo it all ground up, and if models improve, do it better next time.
I exaggerate only a little.
I'm with you on this one, having "vibe coded" some smaller internal tools on GPT 5, and then re-vibed it on Opus 4.6 and 5.5 -- they basically just fixed all of the problems without me doing much of anything other than prompting it to look at the existing code and make it "better".
How much is your budget for tokens?
As long as it's under the budget for X number of senior software devs, it seems competitive.
Pretty much. We're intensely vibe coding something that has gone through so many requirement changes. The code has become very gnarly. I took a stab at basically one prompt rewrite of the whole thing. And it wasn't there, but it was 80% of the way there. and a hell of a lot cleaner.
"Purely AI written systems will scale to a point of complexity"
You have not seen the spreadsheets that accounts run the firm on.
Bloody kids!
I'm sure AI capabilities will plateau any moment now..
Are you sure about this? Yes, there is a stable set, but they are used in all of the wrong places, particularly in places where they don't belong because juniors and now AIs can recite them and want to use them everywhere. That's not even discussing whether the stable set itself is correct or not - it's dubious at this point.
Have you watched Jurassic Park? That story is not about Dinos.
Those design principles it will take us 20 years to learn are just the principles for writing good maintainable, debug-able, understandable code today. Will just take 20 years to figure out they still apply when AI writes the code, too.
No. You can use AI to code this way. I’ve successfully steered AI to implement good architecture by moving slowly and constantly course correcting
Yes but most people won’t.
Many teams did this before AI too. They start faster and end up with hard to refactor or extend code. For example, think of teams that don’t version their API /api/v1, that blocks a whole category of refactoring and extensions. Or teams that have random state transformation routes instead of following restful actions.
Why would it take 20 years to learn? People all around me, in an AI pilled company, have been saying this the whole time,
Interesting perspective. Fundamentally at conflict with the data, science, and 20+ year trends of AI coding systems - to the point of dogmatism. But interesting from a sociological point of view.
is this true because training companies have not been training AI for both performance and brevity (or some other metric like that)? If this becomes a much more serious issue surely they would adjust the training processes
What you're describing really isn't a new problem for organizations. Historically it's been a team of humans not using AI who gets over their skis and they have to have other more capable humans (also not using AI) to bail them out.
I think it will be needless verbose complexity.
I kind of imagine someone having an unlimited budget of free amazon stuff shipped to their house.
In theory, they are living a prosperous life of plenty.
In reality, they will be drowning in something that isn't prosperity.
I don't understand this point of view at all. There's a symmetry that is going entirely unappreciated by most of the comments in the thread: just as I can give Claude X,000 words of text to use to describe the code I want it to write, I can also give it some existing code and ask for X,000 words of text explaining what it does. (Call it, oh, I don't know, a "spec," maybe.)
The explanation, in turn, can be fed back to recreate the functionality of the original code.
At that point, why care about the code at all? If it works, it works. If it doesn't, tell the model to fix it. You did ask for tests, right?
That is where we're indisputably headed. It's not quite a lossless loop yet, but those who say it won't or can't happen bear a heavy burden of proof.
Code is not spec. There is an implementation spectrum.
On one end, you have code that can perform only the behaviour explicitly declared in the spec, but has to be thrown away and rewritten for any new or updated spec.
On the other end, you have code that implements or anticipates a wide range of future possible specs including the given one.
The AI can operate on any point on this spectrum, but it's not very good at choosing. The more complex the software, the more such choices need to be made.
When the number of bad choices reaches a certain critical mass, even a skilled engineer becomes powerless to undo all the bad choices, and even a powerful model becomes unable to reduce it back to a coherent spec.
following along with the amazon analogy...
Some people are mindful about what they get and don't get from amazon and don't die from prosperity. ("you might use AI to increase your prosperity")
the rest of the world eats too much and dies of heart disease/diabetes. ("the rest of the world will flounder more and AI will do more stuff to them than for them")
In their current forms, it's unlikely for a product that actually needs to work.
It's not getting that complex and working with current LLMs.
Yeah, LLMs will have to advance a bit more to really create slop of great complexity.
I've already done a handful of these gigs for early vibecoded products that had collapsed in on themselves. The scope of work was to stabilize the product and only make existing features work.
The issues have all been structural, not local. It's easier to treat it like a rewrite using the original as a super detailed product spec. Working on the existing codebase works, but you have to aggressively modularize everything anyway to untangle it rather than attack it from the top down.
All of these projects have gone well, but I haven't run into a case where a feature they thought was implemented isn't possible. That will happen eventually.
It's honestly good, quick work as a contractor. But I do hope they invest in building expertise from that point rather than treating it like a stable base to continue vibecoding on.
How do you find this type of work??
I've worked with many people over the years. A bunch of product people have struck out to make their own thing now that they can get a feedback loop going. I just keep in touch with people. They know my services are available, so if they have a need they reach out.
The greatest asset in this type of work is genuinely liking people, being good at what you do, and keeping in touch. My email is easily findable for a reason.
So, there is no secret sauce? Just work for people and have a hand at their contact info?
The one part I do wonder is how to "keep in touch". Maybe it's a generational thing as a young millennial (some would call it "Zillenial") but the biggest issue in my networking over the year (cough and the dating scene cough) is ghosting. You think you hit it off, try to follow up the day after, and proceed to never again hear from them.
This is def true but I also wonder if AI models and context sizes and capabilities will scale to keep up and eventually be able to untangle the mess.
My current business plan!
We already know them but everyone is busy throwing them in the trash. It’s all gas and no breaks or handling right now.
But won’t those more complex systems presumably solve more complex problems than the systems that humans could build? Or within a comparable time?
I think it is reasonably safe to assume at this point in the game that these AI systems are increasingly able to reason rigorously about novel problems presented to them, of ever increasing complexity and sophistication.
That sounds so horrible, though. It's akin to people working as COBOL devs because someone has to do it, so they'll get the big bucks. Except I've never heard of anyone who actually likes COBOL and the more I've learned about how mainframe development actually works, the more horrified I've become haha. Dealing with an LLM spaghetti codebase sounds like hell.
There's an LLM for that too...
https://www.hypercubic.ai/hopper
My company and my buddy's company, we're experiencing the same thing. We are trying to fire a SAAS vendor and it's become the hot new project. Now we to these meetings with 50 different people that are allegedly stakeholders, two or three product managers who have already vibcoded their version of something.
Ultimately, if you want to move fast, it's better just to have one engineer vibe coding something. but, that engineer is under so much pressure. Now he's got a legacy mode and another legacy mode because the requirements keep changing. And now there's a deadline in four weeks.
This all could work just fine, but the ungodly amount of attention that this world is getting puts too many cooks in the kitchen, which is always a recipe for disaster.
Sounds like wishful thinking by a human programmer. Probably more likely tools will be rewritten from requirements or the old code will be refactored by newer models.
This is... Not what psychosis means? Being wrong is not psychosis
being wrong and insisting on being wrong is
According to DSM V delusion is a key criteria to diagnose psychotic disorder.
I shut down AI Agent fanatics on the regular. But chop one head off there and two take its place. And I say that as someone working with Claude and Codex daily. While they are both incredibly good at clearly described and defined atomic tasks, application scope makes them lose their minds and the slop ensues.
Saying the _quiet_ part out loud.
I am really looking for more reasoned approaches to AI.
I am very close to using it as a pair programmer, but with me actually coding. I am just so tired of fixing its mistakes.
Isn't going to happen without the regulation hammer being thrown down.
Probably from the EU because they seem to be the sane ones of this generation.
Talking about my own personal workflow. No company has dictated one tl me yet lol.
The race to invent variants of Gas Towns, Ralph loops, pump out videos, blogs, etc. showing off greenfield development with cleverly named agents running in parallel is another case of engineering people diving head first into Resume Driven Development.
Sure there are industry changing things going on. What if you're working on an app thats a decade old and has had different teams of people, styles, frameworks (thanks to the JS-framework-a-week Resume Driven Development)? Some markdown docs and a loop of agents isn't going to help when humans have trouble understanding what the app does.
I work for a small telecom services provider whose current VP immediately set an AI course when stepping on board 6 months ago. Involving AI in everything and every task is now our first priority - across all employee segments, not just us system developers - and leadership is embarking on a program to measure employees' AI usage levels as a means to gauge everyone's individual efficiency. It's like the era of the evangelic crypto bros all over again.
'AI psychosis' is a slop concept.
There's a lot of people writing bad code. With AI being forced top down (with the promise of turning people into 10x-ers), we're going to get a lot of people writing bad code 10x faster.
I really do worry - I especially worry about security. You thought supply chain security management was an impossible task with NPM? Let me introduce to AI - you can look forward to the days of AI poisoning where AIs will infiltrate, exfiltrate, or just destroy and there's no way of stopping it because you cannot examine the internals of the system.
AI has turbo charged people's lax attitude to security.
God help us.
Not security, but I ran into a related supply-chain issue recently. I needed a library to perform a moderately complex task, and found one in the ecosystem I was working with that had been around for a while, appeared reputable, and passed my cursory inspection. So I dropped it in, got the feature implemented, and moved on.
Some time down the line, I discover CPU being maxed out, which is showing up in degraded performance in other parts of the system. I investigate, and I trace the issue to a boneheaded busy loop in this library that no human with the domain expertise to implement the library would have written. Turns out I'd missed one deeply-buried mention in the README that maintenance was being done via AI now, and basically the whole library had been rewritten from the ground up from the reliable tool it used to be to a vibecoded imitation.
Yeah, yeah, sure, bad libraries existed before all this. But there used to be signals you picked up on to filter the gold from the dreck. Those signals don't work anymore.
Preach. No amount if AI will give an administrator an engineer's mind.
Hype & greed are a hell of a drug
I don't think it's helpful to call this psychosis. N Beyond that I don't think it's even irrational.
It is definitely factual that there is a complete paradigm shift in the prioritization of quality in software. It's beyond just AI side effects, and now its own stand alone thing.
There have always been many industries, companies, and products who are low on quality scale but so cheap that it makes good business sense, both for the producer and the consumer.
Definitely many companies are explicitly chosing this business strategy. Definitely also many companies that don't actually realize they are implicitly doing this.
Wether the market will accept the new software quality paradigm or not remains an open question.
Less users can be the cause of less bug reports
Make the most of it. Their delusion is your opportunity.
The hype or psychosis is mainly by mediocre/non expert/middle manager/you name it, especially when a person who never wrote a single line of code suddenly is making a wall of text, and it actually works!? Oh my!!
But in reality, anyone who knows their field and are going after certain specific issue, they will find soon how AI is nothing but an assistant, sure it can help and automate some stuff, but that’s it, you need to keep it leashed and laser focused on that specific issue. I personally tried all high end ones, and I found a common theme, they are designed to find a solution or an answer no matter what, even if that solution is a workaround built on top of workarounds, it’s like welding all sort of connections between A and B resulting in a fractal structure rather than just finding a straight path, if you keep it going and flowing on its own, the results are convoluted and way over complicated, and not the good complexity, the bad kind.
The entire problem is vibe coding is only good for demos, prototyping and finding signs of product market fit without actually releasing a product into the market.
You should not release a product into the market unless you have a good enough product that can keep you and your client compliant, safe and secure - including not leaking their customer info all over the place.
Prompt injection risk, etc. are massive for agentic AI without deterministic guardrails that actually work in practice.
Stop testing in production if you're shipping in a regulated industry. Ridic!
If you're not technical, you can get someone who is after signs of p-m fit, demos, but BEFORE deployment. This is common sense and best practices but startup bros dgaf because they're just good at sales and marketing & short term greedy.
Comical.
The Twitter post doesn’t even document some of the most psychotic things that are happening.
I saw this first hand at a company, and I think this is what happens when you combine FOMO with an utter lack of industry best practices. No one knows where they are going, but are convinced they are not getting there fast enough.
What's more, the only people they talk to about it are others at the same company. There is no external touchstone. There are power dynamics from hierarchy. No new ideas other than what is generated within the company. In other circumstances, this is a textbook environment for radicalization.
I would encourage all leadership to take a deep breath. You have time to think slow.
Totally unrelated pet peeve of mine, I hate when people write this: "MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery)".
You first use the full words and then introduce the acronym that you're going to use in the rest of the text: "Mean Time Between Failures (MTBF) vs. Mean Time to Recovery (MTTR)".
With the latter, readers understand the term immediately, even if they don’t know the acronym. And they don't have to read these weird letters before getting the explanation.
Amazing how the dev community is suffering from a similar inability to approach the subject of real world AI efficiencies and business benefits. I don’t think it’s helpful to accuse the other side of psychosis. It disqualifies any data or experience they bring to the conversation.
It is not the dev community writ large, it is a particular archetype among forum users, particularly common among forums with upvote mechanics
It seems the diagnosis of psychosis is too quick: it seeks to reestablish the frame of expert for the developer identity that is being replaced by it.
“It feels like entire companies are deluded into thinking they don’t need me, but they still need me. Help!”
The broad sentiment across statements of this “AI psychosis” type is clear, but I think the baseline reality is simpler. How can you be so certain it’s psychosis if you don’t know what will unfold? Might reaching for the premature certainty of making others wrong, satisfying that it might be to the ego, be simply a way to compensate the challenges of a changing work environment, and a substitute for actually considering the practical ways you could adapt to that? Might it not be more helpful and profitable to consider “how can I build windmills, ride this wave, and adapt to the changing market under this revolution” than soothing myself with the delusion that all these companies think they don’t need me now, but they’ll be sorry.
The developer role is changing, but it doesn’t have to be an existential crisis. Even though it may feel that way — but probably it’s gonna feel more that way the more you remain stuck in old patterns and over-certainty about how things are doesn’t help, (tho it may feel good). This is the time to be observant and curious and get ready to update your perspective.
You may hide from this broad take (that AI psychosis statements are cope) by retreating into specific nuance: “I didn’t mean it that way, you’re wrong. This is still valid.” But the vocabulary betrays motive. Resorting to clinical derogatory language like “AI psychosis” invokes a “superior expert judgment” frame immediately, and in zeitgeist context this is a big tell. It signifies a need to be right, anda deeply defensive pose rather than a clear assay of what’s real in a rapidly changing world. The anxiety driving the language speaks far louder than any technical pedantry used to justify it, and is the most important and IMO profitable thing to address.
Pointing out the obvious.
A lot of companies have been under AI psychosis for years and will be forever.
If you don't use it you lose it, and a lot of people are losing it..
If you know these things you can take them into account while driving the AI.
Sorry, I don't buy your argument
I feel in a really weird position where I both really dislike what AI is doing to the experience and practice of writing code, to the point where I want a job doing literally anything else besides using the computer, but also think that these tools are extremely powerful and only getting better.
I think Mitchell's point is well taken -- it's possible for these tools to introduce rotten foundations that will only be found out later when the whole structure collapsed. I don't want to be in the position of being on the hook when that happens and not having the deep understanding of the code base that I used to.
But humans have introduced subtle yet catastrophic bugs into code forever too... A lot of this feels like an open empirical question. Will we see many systems collapse in horrifying ways that they uniquely didn't before? Maybe some, but will we also not learn that we need to shift more to specification and validation? Idk, it just seems to me like this style of building systems is inevitable even as there may be some bumps along the way.
I feel like many in the anti camp have their own kind of reactionary psychosis. I want nothing to do with AI but I also can't deny my experience of using these tools. I wish there were more venues for this kind of realist but negative discussion of AI. Mitchell is a great dev for this reason.
I've never had more fun coding, but the key is actually still writing the code yourself. The LLM has terrible judgment but an encyclopedic knowledge and the ability to pick out important details in a sea of information. Their worse use is producing code, but somehow that gets all the energy. Being an LLM babysitter is energy draining and you feel less and less in control. No job is worth being miserable doing something that you used to enjoy.
What do you use an LLM for then? Or you’re saying just don’t use it at all?
I've used both ChatGPT and Claude, they seem interchangeable for my needs. I only use the web prompt interface except for the rare occasion that it is helpful for it to have the context of my entire project. I think less is more when it comes to LLM interaction, but sometimes they are exactly the right tool for the job.
What do you use it for then?
“right tool for the job” - what job exactly, why so mysterious?
I didn't realize you wanted that information too, I could probably bore someone to death talking about it.
Planning: I often ask it to help me plan an approach if we are dealing with something I don't have a lot of experience with, most recently working with the DOM. If there is a library or an API that is new to me, I ask for an overview and run my plan by it for comments. Feed it the documentation and it is like talking to author.
Coding: I have a pretty reliable sense for when a section of code that I want to write is obvious enough for the LLM to one-shot based on the other code in the file, and on those occasions I call in completion. I do this with code that I can verify at a glance.
Analysis: If I have any uncertainty at all about the code I've written, I run it by the LLM to find issues. Out of all the other uses, I think this is the most productive and time saving. If I run into a bug and I'm stumped, I show it the section of code. I'm amazed at how good it is at finding mistakes.
I'm working solo as a full stack developer coming from a different background, so I sometimes find myself out of my depth. Having access to the breadth of knowledge that an LLM brings and its attention to detail has been game changing. I've tried a couple agents and configuring them to work competently seems like a rabbit hole, and I like the tight control over the context that chatting with the web prompt interface brings. It seems like half the value is putting into words my intent, it forces me to have a cohesive understanding myself. It is like rubber duck debugging where the duck can actually talk back and sometimes provide the critical part that I'm missing. I have it speak like a pirate which is just for fun but sometimes the sailing metaphors that it uses are really intuitive.
Thanks for the comprehensive explanation!
I've been working on a C++ backend for F# and while I'm very familiar with F# and it's AST I barely know C++. The amount of time I save being able to ask things, check my understanding, get design patterns, and paste issues I'm having for a fix is insane
I ran into an issue where I was getting a segfault and everything looked right in the debuggr, including expected values near the segfault. Turns out I wasn't using placement new somewhere I needed, and the data for the object was getting copied but not the vtables. I have no idea how long it would have taken me to figure that out on my own because the segfault was coming from so far away
I haven't had the opportunity to use LLMs much for coding since I'm not working right now, but I can second how much of a boost just getting specific answers to my questions instead of reading tons of whatever online searches return is.
Rubber duck that talks back is a nice way to put it
This is more or less how I use Claude and Kim 2.5 via Kagi. If I just let it spew out code I have no idea what it does and no interest whatsoever to try to comn through it all. But when I have need to ask about syntax or correct use of library function etc. - I’m learning C++ - and can’t grok the docs, it can be incredibly helpful. Also is great at finding bugs.
I think of it kinda “very knowledgeable dumb person” - it knows everything but understands fuck all (although it can appear to do so just by breadth of information it has). If I can formulate a question in a way it gives me the correct info it helps me to conceptually understand the problem better then filling out the blanks. Often I figure out the answer to my question just by writing it down without needing to prompt it, so speaking rubber duck is very apt way to call it.
working in a large codebase I use Claude for code understanding and the code reviews from Macroscope have caught bugs for me a bunch of times. Usually if I use claude it’s for refactoring a and source to source transformations that would be too confusing for me to figure out how to do with e.g. ast-grep, but that I can prompt in a minute or two and then have claude work through it. It’s stuff I could do without LLMs but it’s less effort to use them. I don’t let it write new code, because it decays the process of programming as theory building.
not the person you're replying to, but someone who agrees with the gist of their message - I personally use Claude Code as a better Google search for debugging and syntax.
It used to be "oh, why am I getting an error on line 352, let me google the error message and wade through Stack Overflow answers" now it's "Claude, why am I getting an error on line 352? Ah, it's because $REASON, let's see if that fixes it, yes, thank you."
Obviously reading the official documentation is very useful, but sometimes you can't find anything that relevant to your exact use case, and forums are also very useful, but it can take hours or even days to get a reply to question when the LLM can do it in like a minute.
In my experience, its the opposite. The AI is very good at writing code, but it is unreliable at any kind of design. I use it as a fancy form of autocomplete. I give the broad strokes: "add a method here and change all but this one caller to use the new method", "Apply this design pattern here for this change but don't do this other thing". It completes the task reasonably well and sometimes even remembers to run the code formatter and check that tests pass.
If I ask it to me produce a design, I'll almost always end up with something unworkable or inefficient.
Though if you push it hard enough then it can sometimes give you a good description of what existing code does and how it does it (which can be easily verified).
Can you pro-AI people at least agree on what is it even useful for?
Every thread is endless back-and-forth between the "AI works great and vibe coding is the future" and "no, AI works great as long as you don't vibe code" camps.
AI is in like year 4 of being useful for the task. And it’s changing rapidly. We don’t know what it’s good for yet!
But pick a technology used to write code and you’ll see many of the same things. Broad unit test adoption happened more then 20 years ago and people still can’t decide what it’s good for or even if it is useful.
That’s because it depends on the circumstances of the problem and the person solving it.
So no, we can’t decide exactly what it’s good for and if we could there would be exceptions. But that’s not an indictment of any tool.
So the solution is to wait and not use AI until people have spent all their money and actually figured out what the real use cases are. Good to know.
Is that your approach to unit tests? If so, sure.
Is it so hard to imagine that people who find LLMs useful might have different perspectives on it? If you put a lot of energy into being against something you might think that everyone who doesn't agree puts the same energy into being in favor of it, but that's not real life.
Can developers agree on what the best programming language is? The best framework? The best OS?
AI is a diverse technology and will work differently for everyone, just like JS is a really, really bad technology for many, yet widely used.
On top of that it (as coding agent) is brand new, just a few years old. Looking at the history of technology it will probably take a few decades for it to settle into some sort of stability. Which does not mean it does not provide value during this time of exploration, however there is no trodden path leading you there.
Fancy autocomplete is a solid use. I use the same pattern replication technique whenever the situation allows for it. It's great for following your lead, but unreliable at taking the lead. Even if it was more reliable, I won't have the same working knowledge of the structure it created.
My biggest complaint about AI is getting it to lock onto current or specific information has been darn near impossible. Its definitely there in its training data but for the life of me I cannot get it to stop bleeding long outdated or external information into its responses.
I find in these cases the best option is to provide references, docs, etc. If you'll need them regularly, save them to Markdown files.
Bingo. Claude can bang out nearly correct code when I give it an idea, but it doesn't have the idea and repeatedly misjudges both how much work remains and what kind.
On the other hand, I don't know all the ins or outs of macro expansion in yaml at compile time or when and where macros run, enabling us to conaume their results elsewhere in the yaml. Frankly, if I had time, I'd happily spend time on that and learn more about it. I don't, though. Claude knows and does the guessing and checking. So I provide the concept and it translates into a horrible soup of yaml. Clearly I'm able to press forward with ignorance, which is dangerous. There's a real risk that I'll wind up with the kinds of unhealthy work that worries the author of the tweet.
After turning off my brain a few times and ending up in a place I don't want real fast, I am learning to ride this dragon.
And, you are right - use it as a fast typer, not a fast thinkier.
And for those who claim that AI is a good code writer - wrong. It can OUTPUT a wall of code, but it's overeager to flood you will legacy code on arrival to "solve a problem". It's harder to write LESS code, which is still the goal (even more true today).
That's a great way to put it.
So now the AIs will do more of that, at superhuman speed.
We'll just quickly learn what we've been trying to do for decades, while also treading water in floods of more code than has ever been written before? And some of the motivations to write correct code are being deflated - "just vibecode it again and see if the bugs disappear, it only took a week and $200."
I think the commenter was referring to formal verification with "specification and validation": have the LLM emit formal proofs about invariants etc.
Currently the bugs are found by people using LLM's but aren't the developers. As more projects start getting access to compute, they can run those LLM searches for bugs themselves, and can simply prevent shipping the bugs.
I'm surprised no one has tried making any statistical analysis of bug densities, and "bug authors" in an attempt to identify untrustworthy developers, regardless of intent. Given a dataset of authors and prior bugs, it may help find more bugs by tracking their pull requests with higher scrutiny...
Some people may end up with an eternal stain if they've been taking money to submit vulnerable code to code bases...
You're using psychosis wrong. My literal reality is my entire industry trying to use Ai as an excuse to payoff hundreds of thousands, to millions of American engineers in lieu of outsourcing work overseas. It's having hostile promots to use AI that never truly go away (if you're even given an option to turn off the prompt). It's seeing an emerging generation completely stunted because AI's best use is to cheat the education system and ruin the youth's critical thinking. It's looking in apallment at proposals for data centers that take more energy than the state actually has.
And while you can try to call these exaggerations, you're falling into the very psychosis of this article if you want to deny this reality as a whole. "but the tech is making us so productive" is not a valid justification to literally collapse human society as we know it.
Also "reactionary" haha
"looking on appalled at..."
Not FANG, but I work at a company that operates some infrastructure at scale. What I've seen is after we've rotated through a number of different tools, in different pilot groups, eventually converging on tool X (a custom, internal wrapper around opencode).
Now every "working session" like meeting, at the team or dept level, has been around how to use tool X. Tricks using tool X. Problems using tool X. I can't help feeling if we had spent the same amount of time building up core knowledge/contempencies around say, design patterns, networking, specs, we'd be in a better place for building.
Instead we are going to have a few thousand people who know a tool really well.
Maybe this is what will turn software engineering into an Engineering field.
Right know, prompters are setting up whole company infrastructure. I personally know one. He migrated the companies database to a newer Postgres version. He was successful in the end, but I was gnawing my teeth when he described every step of the process.
It sounded like "And then, I poured gasoline on the servers while smoking a cigarette. But don't worry, I found a fire extinguisher in the basement. The gauge says it's empty, but I can still hear some liquid when I shake it..."
If he leaves the company, they will need an even more confident prompter to maintain their DB infrastructure.
Oh man, I think you may have touched the third rail here.
My first job out of high school was as an AutoCAD/network admin at a large Civil & Structural firm. I later got further into tech, but after my initial experience with real Engineering, "software engineering" always made my eyes roll. Without real enforced standards, without consequences, it's been vibe engineering the whole time.
In Civil, Structural, and many other fields, Engineers have a path to Professional Engineer. That PE stamp means that you suffer actual legal consequences if you are found guilty of gross negligence in your field. This is why Engineering firms are a collective of actual Professional Engineer partners, and not your average corporate structure.
The issue is that in software dev, we move fast, SOC2 is screenshot theater, and actual Engineering would slow things way down. But, now that coding is fast, maybe you are correct! Maybe vibe coding is the forcing function for actual Software Engineering!
___
edit: I just searched to see if my comment was correct, and it turns out that Software PE was attempted! It was discontinued due to low participation.
https://ncees.org/ncees-discontinuing-pe-software-engineerin...
Note that other types of engineering are also often vibes based. The mechanical engineering for a rocket engine is extremely rigorous but the engineering for an injection molded housing for a cheap cell phone is a lot more about following a few heuristics and getting it out the door. Even in robotics where I work, it’s mostly about making parts that pass whatever acceptance tests you come up with. In civil engineering and aerospace failure costs human lives and millions or billions of dollars. In robotics maybe you have some machines fail in the field but in many instances you have one overarching safety system and many of the parts are irrelevant to that. The camera housing for example. So no paper trail or mathematical design validation is required to prove you designed it right. Often those are desirable but if you just manufacture it and test it a lot you’re probably fine.
This was something I noticed in my early career in mechanical engineering and later doing PCB design and software for robotics. It’s easy to find firms that just need adequate parts without the professional certifications or ass-covering calculations of other engineering fields.
All this to say, it’s not just software versus the rest of them. From my position, civil and aerospace seemed more like the exception while much of the rest of the engineering world is more vibes based.
Sure, but as "software eats the world," maybe it should be the most formalized of all Engineering, as it runs everything...?
Probably not. Some things are critically important, but most things just don’t matter that much if they break or degrade some.
Perhaps this will make a comeback when the need arises to distinguish between actual software programmers and prompters.
This is actually not a bad exam to administer: https://ncees.org/wp-content/uploads/2025/01/FINAL_PE-Electr...
In the Civil & Structural world, there is no greater honor than to be on the standards committee.
I hope that this becomes a thing in Software Engineering.
Imagine how much money will go into lobbying for some particular AI to BE the standards committee.
IEEE SWEBOK tried to and is generally considered not to have achieved what it set out to do:
https://en.wikipedia.org/wiki/Software_Engineering_Body_of_K...
https://news.ycombinator.com/item?id=41907412
See if you can find anyone outside of PE relying on it. ACM withdrew from it in 2000.
Yeah, did they ever put out recommended study materials too?
What makes it a profession is not just the certification, it's the burden of responsibility for consequences. Your lawyer, accountant, and real engineers carry "we need insurance for this" level of risk in their work, all the way up to "can go to prison for getting things really wrong".
Until and unless software is held to that standard, software will never be engineering and always just a craft that can be performed to any or no standard.
Eh writing software for healthcare, or aircraft or self driving cars is more rigorous than an EE working on industrial lighting or toys.
Im sure for the most part, engineers in physical space deal with the same kind of tradeoffs software engineers make, where you try your best based on industry standards, personal past experiences without some way to prove what youve done is right
That’s a relatively small field within the software industry.
Most of the work being done (adding new fields to CRUD apps etc) is glorified clerical work, where the people doing it are rightfully fearful of being automated out of existence by AI.
So it sounds like it was fine? Why would this prompt (haha) a change in their approach to things?
Now imagine if you’re one step removed. You don’t see the cigarettes, smell the gasoline, nor see the fire extinguisher gauge. You only see the servers running business-as-usual. Those “engineering” guys are always drama queens, you think. We have processes and fire extinguishers when shit hits the fan, right?
That’s basically every M2, and many if not most M1s, in the last 10 years. So fuck it. Why does any of it matters?
... If it's been like that for 10 years then it can hardly be blamed on "AI".
Yes, you can’t blame AI for it. But it was a self limiting system. You couldn’t just go to a fresh college hire and ask them to do deep surgery on the entire stack. You would go to your very senior engineers for that. Those senior engineers will push back on some stuff then in the push-pull cycle you would have to settle at some middle ground.
With AI I’m seeing managers literally get an intern, ask them if they can change fundamental assumptions of a system, give the intern claude 1M window, have the intern ready with a 37k line PR in an afternoon and then go ping a senior engineer if they can “take a look”.
As a junior dev there is this pressure to produce code, add features, and investigate bugs within unprecedented time period. I know whole code base is fking up but i will still add that feature or do a sloppy bug fix without digging deeper.
In my experience, AI really lowered the bar for bad code in the name of delivering faster.
I have seen people write highly complex code where all the complexity was not necessary. Think: deep unnecessary branching, pointless error handling and retries which make no sense in our context, hand-coded parsing using regexps, haphazard data flow, functions which seem purely computational but slyly make API calls, pointlessly nullable model fields, verbose doc comments which describe the implementation instead of the contract. I could go on.
The worst part is, even when "prompted" by bad coders, it works in the end. Even has tests (ostensibly mock-ridden, a pet peeve of mine which always falls on deaf ears). So I cannot reject the PR without being an asshole.
I am no luddite. I make heavy use of AI, with all the skills / AGENTS.md / style guides and clear specs, then review every line of code, prefer testing with minimal mocking. I'd even say with right prompting, it can write better low level code than me (eg: anticipating common error conditions).
But my biggest fear about AI is how it enables normies with little to no understanding of CS principles to produce code faster which looks correct but slowly poisons the codebase.
I have a friend, smart guy, who is writing web services and “connecting them together” for a large firm; he has absolutely no programming experience.
Talking to him, he told me he couldn’t even reverse a string. He is at once many times more valuable than ever before to his company, but also far more dangerous than ever before.
This is what fascinates me. I have a friend, also a smart guy, who has made it to the point he’s at by being a kind of solutions expert. He’s an IT guy, basically. He’s very technical but has never claimed to be a software engineer. He’s writing software with Claude now. The other day he sent me a screenshot of some other team at his work asking him to shut off something he made that was brutalizing an API of theirs. I asked him if he had ever heard of a 429 or exponential back offs. He said no. How do you meta-prompt for that without knowledge?
You can create an agent in Claude with the role of Technical Lead / Architect and have it review your code. That depends on your agent specification. Just have ChatGPT generate that first.
If you get the logs you can feed them in and ask for improvements, that sometimes helps.
But even then, you have to know to do that. It feels like a bit of a turtles all the way down situation, no?
He's "smart" but he chooses to be in a business where he's presumptively willfully ignorant of the fundamentals (since he surely should be able to learn to reverse a string if he wanted to learn)? He doesn't have a more lucrative opportunity available? Or does he somehow have a skillset that makes him able to "connect web services together" by prompting AIs in ways that other people (including ones who can reverse strings, etc.) couldn't?
This form of being "smart" is a bit difficult for me to comprehend, I must admit.
I strongly agree with this. Suddenly with the mass adoption of LLMs there are so many smart, yet naive people out there willing to toe the line. Why these smart people couldn't bring value in a million different other ways is, of course, left unsaid.
They're not even trying to dress up these bullshit stories anymore. In truth it doesn't matter if you believe it. So much buzz is people just talking to themselves out loud.
Well that’s not his primary job. It’s an extra task he’s doing at his job. At a non-tech company with small/horrific engineering components, someone in the business who can do any programming (or vibeing) is indistinguishable from magic.
When I read the discussions about AI making code worse I keep bringing the same argument: people made bad code even before AI. Average coder is barely functioning and that's a fact.
And we were safe from them because they couldn’t produce a mountain of code every day. But soon many places will be buried under a planet of unmaintainable code. It’s adding friction and operational cost and often not adding value.
People could, however, learn to not make bad code. LLMs are incapable of that feat because they do not have any understanding or ability to reason. They are strictly worse than a human.
As others have elaborated, the problem is empowering them to ship mountains of bad code;
And yeah, many semi-technical M2s or even M1s can't distinguish bad code from good code, or worse bad architecture from good; this is golden time for those who are willing to sacrifice the future for present. Just burnnn'em tokenzzz.
I would've believed that 6 months ago, but not now.
If you have a good codebase with proper rails, hygiene and architecture, AI will produce better code than most engineers out there.
People forget that 90% of the field has always been charlatans barely able to implement a fizz buzz or go much beyond trial and error googling.
I'll say even more. I'm in the 10%, and it's increasingly clear to me that AI writes in minutes code that's better than mine.
Even stellar and respected OSS engineers are nowadays leveraging AI and guiding it less and less everyday beyond giving indications of what kind of data structure they may want for a complex problem or the kind of architecture they are looking for.
In any case, I don't like this field anymore, I have no joy from it, way too much work, way too many changes a human can cope with both on product and technological level (not even counting AI and its tooling itself). The interesting parts of thinking an entire afternoon or week experimenting to get that design right disassembling the pros and cons are gone.
Even if you want to do that, it's just faster to launch 6/7 worktrees with the different ideas and judge the results. But you don't get as intimate with the problem and the amount of information is way more than you can process.
Forgive my ignorance, but if the corpus of coding data was always 90% bad, isn't that the same data being used for training LLMs? How are they magically any better than that average?
Because LLMs are not stochastic parrots.
Programmer: "What is this slop that I found in your code?"
AI: "I LEARNED IT FROM YOU, DAD!"
They aren't. The guy you're replying to is just hyping them up based on nothing.
I'm hand rolling a project right now because even frontier models I use bloat things beyond comprehension. Because I'm intimately familiar with the domain, I know the shape of things, how the data should flow, and so on, and if l even if I spec it clearly AI will write 2x to 5x the amount of code necessary to make something work.
"beyond comprehension" is a good way of putting it. I've been genuinely baffled by some of these AI designs - why any intelligent thing would write >10 lines of bloat for what should be a one-liner.
As Anthropic's drones say: treat Claude as your genius coworker. Don't think yourself, don't judge, the machine must know better than you. It is the genius, after all, not you.
Okay, so Ai is completely useless in my industry. Got it.
Proper rails, hygiene and architecture need to be actively maintained, they don’t just continue to exist in a developing codebase. Historically, a small proportion (the 10% as you say) had a disproportionate amount of influence on coding standards. When they can no longer keep up with that ongoing maintenance, which we’re seeing with the increased pressure to ship code, the hygiene will regress. We’re riding the tail of all the engineering practices we’ve developed as an industry.
This is what I’m seeing, anyways. Junior engineers are being rewarded for shipping so much code, it’s impossible to evaluate it all, and subtle changes in existing patterns are slipping through. Eventually all those subtle changes transform the rails.
This means you take less time reviewing code than it took for the machine to churn it out. All that code must be a ticking time bomb.
90% of software engineers are not charlatans, and it's convenient that you at confidently place yourself in the "10%"
So many fallbacks. So many function_exists. So much pointless type casting. I swear it’s like the system prompt is designed to waste as many tokens as possible.
Ironic: my value as a programmer now comes not from my ability to write code, but my ability to delete the useless fluff that AI wrote.
This is a social problem that I had thought the industry had solved a long time ago.
I agree 100%. At work I'm teaching software engineering principles to system administrators and they, too, often use AI without thinking. Then it's on me to provide feedback on the PRs they barely read themselves. Not a lot of fun but slowly but surely they're learning.
Yes, you absolutely can. And you should. Try to teach them lessons and what patterns to watch out for, then tell them to put those insights in their CLAUDE.md, so that their agent becomes better, too. You can also tell them to just copy your own CLAUDE.md, like I did: https://github.com/codethief/ENGINEERING_PRINCIPLES.md
Stay strong!
I think it’ll be the opposite. Maybe it’ll be what will eventually cement the field as “talent” based field. Just like it was difficult to quantify what makes a flute player better than another, how good your are at endlessly prompting a blackbox machine would be the only measure. The engineers of ol’ whoe developed kernels and drivers would be thought of as the “crazy people who put the flute against their temple to tune it” LOL. we don’t need people like that. You can just buy a flute tuning device. who gives a fuck? Can you make the next “Shake it, Shake it”?
I work at software in a medical setting. We are piloting an integration with a startup for measuring [some bodily variable relevant in ICU setting]. They are obviously vibecoding (docs are telling) and their API is failing in unexpected ways that they are not able to resolve. I am just waiting when this are going to harm somebody.
Don’t worry, some medical professionals are also delegating their thinking to LLMs. No need for the software middleman to cause harm.
This is the pattern you will see when medium-successful ignorant people take o ver a system that was based on some kind of standard.
You can see the same approach is taken by Trump and other people.
“You have TDS!! He is actually doing good. He doesn’t follow rules because the system is rigged etc.”
These arguments border on religion because it is predicated on you believing their ignorant point of view in the first place.
Engineering and science is built on rigor and empirical evidence, it is not built by scammers/businessman/ignorant-people/politicians because that is just not how it works
This reminds me of Rich Hickey’s “Simple Made Easy” and his approach in making Clojure.
Even before LLMs generating entire programs, complex frameworks allowed developers to write the initial versions of programs very quickly, but at the cost of being hard to understand and thus hard to debug or modify.
Some of us are betting that the AIs will always be smart enough to debug, maintain and modify the programs written by AI, no matter how convoluted or complex. I’m not so sure.
Ah, a cognitive Moore’s law, so to speak.
That people don't realize full test coverage just means every line is hit, not that everything is correct is always funny to me. (I don't view as an argument against tests, but with AI it's especially important as if you're aren't careful it'll be very happy to make coverage that is not quite right.)
Correct. Tests don’t tell you the code works. They tell you that something changed that impacts the test since the last time it did work.
Just talked to an exec yesterday about their multinational company, where the newly-installed CEO just came in with "everyone needs to be using AI" and "we should be doing everything with AI".
I cautioned them that this a terrible idea -- you have business people who don't know what they're talking about, and all they know if "if we don't 'do AI' we'll be left behind because our competitors are 'doing AI'" (whatever tf "doing AI" means).
Yes, LLMs are a great tool. But they're not like some magic bullet you stick into everything. Use it where it makes sense, and treat it like you would other tools.
You make "doing AI" some kind of KPI in your org, and you're going to have people "doing AI" amazingly (LOC counts! tokens burned! tickets cleared!) while not actually being more productive, and potentially building something that is going to come down on your head for the next team to "clean up the AI mess".
Is he talking about github?
I'd like to chime in and mention that its really obvious how to RL a coding agent to get the human addicted asap. and its also clear that there's a ton of $$$ to be made by doing this. therefore its done. the only LLMs I use are the ones I run locally because i know they aren't RL'ed for that metric (no incentive for the company that made them to make their open weights models addictive)
Interesting angle, didn't think of this. How do you think/find that current tools are optimized for being addictive?
I think there's a few things, but its a little subjective and its more about the style the ai uses when doing these than the actual specific behavior:
- Nuggesting improvements to the code after finishing the task you gave it, very irritating when the improvements were obvious and the ai didn't implement them on its own
- Not trying very hard when implementing something, leading to bugs, which leads to more tokens used (this behavior can be incentivized and learned with RL)
Since its a known fact if a user continues a session after the LLM says something, its not hard to train against this. The least efficient way to do this would be to GPRO directly against the user base and try to get as many people talking to the AI, and with OAI having a billion monthly active users the least efficient method would work really well for them.
Hard to have sober talk about this since a lot of discourse is AI psychosis vs. AI naysayers. Does software quality seem to have taken a jump in the past few years to anyone? Not to me, seems to be getting worse. Think that's a decent signal. Can tell you I'm dealing with a non-technical VP who loves blast submitting vibe-coded PRs and while there's some quick wins, overall quality is bad, and we had our first real production outage that Claude one-shot caused but could not one-shot solve.
There's an acceleration of current known processes that is being referred to as agent speed (vs human speed). But this is purely a mechanical effect. There don't seem to be augmentive cognitive effects. "AI has invented this revolutionary algorithm/workflow/architecture" is an article title you'd expect to see pop up quick, and often.
I have respect for Mitchel and I’ve spent a good deal of time trying to think of ways to justify his message. I can’t. Either I am missing a big piece or he is worrying about something that comes naturally as more software gets developed (and sooner).
In any case, this is what blue-green deployments and gradual rollouts are for. With basic software engineering processes, you can make your end user experience pretty much bullet proof. Just pay EXTRA attention when touching DNS, network config (for core systems) and database migrations.
Distributed systems are a bit more tricky but k8s and the likes have pretty solid release mechanisms built-in. You are still doomed if your CDN provider goes down. You just have to draw a line somewhere and face the reality head on (for X cost per year this is the level of redundancy we get, but it won’t save us from Y).
The one thing I hadn’t mentioned - one I AM worried about - is security! I’ve been worried about it from before Mythos (basic prompt injection) and with more powerful models now team offence is stronger than ever.
Yeah. The same processes that allow corporations to outsource their software to barely qualified 3rd-world body shops are the processes that allow you to deploy AI-generated code of unknown quality.
There’s this delusion that if we somehow write enough tests that we’ll expunge every defect from software. It’s like everyone forgets that the halting problem exists.
Psychosis means inability to distinguish the real from the not real -- delusion. I don't think the article describes that, at least not in a literal or clinical sense. The author lifted a term usually applied to people who fall in love with chatbots and applied it to the context of software developers not understanding AI coding tools, and the limitations of those tools.
AI coding swept over the software industry faster than most previous trends. OOP and its predecessor "structured programming" took a lot longer. Agile and XP got traction fairly quickly but still took longer than AI -- and met with much of the same kind of resistance and dire predictions of slop and incompetence.
AI tools have led to two parallel delusions: The one Mitchell Hashimoto describes, and the notion that we (programmers) knew how to produce solid, reliable, useful, maintainable code before AI slop came along. As always with tools that give newbs, juniors, managers some leverage (real or imagined) we -- programmers -- get upset and react to the threat with dire warnings. We talk about "technical debt" and "maintainability" and "scalability."
In fact the large majority of non-trivial software projects fail to even meet requirements, much less deliver maintainable code with no tech debt. Most programmers don't know how to write good code for any measure of "good." Our entire industry looks more like a decades-long study of the Dunning-Kruger effect than a rigorous engineering discipline. If we knew how to write reliable code with no tech debt we could teach that to LLMs, but instead we reliably get back the same kind of mediocre code the LLMs trained on (ours), only the LLMs piece it together faster than we can.
With 50 years in the business behind me, and several years of mocking and dismissing AI coding whenever someone brought it up, I got dragged into it by my employer. And then I saw that with guidance and a critical eye, reasonably good specs, guardrails, it performed just as well and sometimes more throroughly than me and almost all of the people I have worked with during my career. It writes better code and notices mistakes, regressions, edge cases better than I can (at least in any reasonable amount of time).
AI coding tools only have to perform better -- for whatever that means to an organization -- than the median programmers. If we set the bar at "perfect" they of course fail, but so do we. We always have. Right now almost all of the buggy, insecure, ugly, confusing software I use came from teams of human programmers who didn't use AI. That will quickly change and I can blame the bugs and crashes and data losses and downtime on AI, we all can, but let's not pretend we're really losing ground with these tools or that we could all, as an industry, do better than the LLMs, because all experience shows that we can't.
If we are just talking about writing code, I believe LLMs are indeed better than the median programmer already. Where they fail, however, is in the tasks that require proper reasoning.
At a micro-scale, you can simulate reasoning with token prediction, but I think this breaks at a higher-level (e.g. architecture.)
I believe that, given enough complexity, a purely vibe-coded project will always reach a point where each new feature breaks five other ones. So qualms about "maintainability" are important.
Maybe we'll reach the point where LLMs can deal with the complexity of most codebases without human guidance.
But it doesn't change the fact that this limitation is inherent to LLMs: chain of thought is not reasoning, only the illusion of it.
Right, and the important part here is that it did so with your guidance and taste, which have both been honed by years of experience.
Also, potentially a good band name in there:
“very resilient catastrophe machine”
What's the historical context for this MTBF vs. MTTR reckoning?
If you optimize for MTBF, you optimize for it to be a long time between failures. You optimize for the system not going down in the first place, but when it does do down it might be Pretty Bad.
If you optimize for MTTR, you don't care how often you go down and instead optimize your recovery time to be as short as possible.
The concepts are pre-computing.
Not the GP commenter, but I'm still struggling to understand how this relates to the AI world, or perhaps more importantly, what the historical context was. Did people end up switching to MTTR optimization over MTBF optimization? If so, is the implication that the recovery times got lower but software instability went up as a result?
MTBF = optimizing quality (reliability, uptime, correctness) of AI product
MTTR = optimize the ability to correct failures when they occur.
He's describing leaders who believe quality no longer matters because any faults or deviations can be corrected so quickly that it doesn't make any sense to waste time on quality.
Yes that’s very correct. The way I think of it, MTTR is easier to measure and manage as a manager. MTTR is all about “operational excellence”. Basically, when shit hits the fan, how good are we at figuring out what caused it and how to fix it. That’s a muscle that you can train, the script goes:
- What alerts are we missing that could have helped us catch that earlier?
- What dashboards could we have had to help diagnose the issue quicker?
- What Ops tools could we have had to help mitigate such issue quicker?
- What extra logging/metrics/telemetry could we add to help us catch this quicker?
- What “safe deployment practices” could we have employed to avoid/improve this?
- what processes could we enforce to facilitate all of that?
Rinse and repeat that few hundreds or thousands of times while mounting MTTR KPI and you will see that number improve. Most likely through your team “gaming it”
MTBF is much, much, tricker to measure or “manage out”. It’s about “excellence in engineering” which is not measurable nor controllable. You want a random feature X. Your team tells you it’s really not how the system works, and they want few months making the change slowly while observing the system. But you don’t want just X, you want X, Y, Z, W, V, Q, A, B, C, D, all the way throw AAZZW12. So you tell the team to go fuck itself.
Same grifters optimizing for MTTR are now pushing even more reckless use of AI, because “accidents will happen anyway, so we need to prioritize speed”.
There are concerns that AI might/will make mistakes. Instead of optimizing for producing perfect code, they think that AI can fix bugs as fast as it produces code and are optimizing for MTTR. Sounds like decision made by people who don't write code regularly, as there is this Architectural drift that happens where you are no longer aware of what's happening in your codebase. As a junior guy I so want this to happen.
To give a timely example, think GitHub and what its leadership is thinking/optimizing for. Do you care if you’re down once or twice a week vs how long those down times are? What’s the KPI you’re managing GitHub with?
Current (and by current I mean the last 4-5 years) they only cared about MTTR. That was probably the only metric they measured and cared about. When a system went down it fired an LSI “Live Site Incident” (as opposed to a CRI “Customer Reported Incident”). At the time you grilled your team. Eventually you come to the conclusion that an LSI should only be measured by MTTR. MTBF is meaningless because MTBF limits your “ship new features” velocity.
You might scoff at GitHub and “ship a new feature” concept in the last 5 years, but if you’re an enterprise customer you’d know how much nonesense they shoveled out in the last 5 years. Absolute insanity of “what the fuck” type feature because customer X who is paying $$$ is asking for it type features.
Before the cloud, people were trying to reduce the mean time between failure (MTBF) essentially trying to prevent a thing from failing. With cloud, people are trying to recover as quickly as possible (mean time to recovery) accepting that things will fail —- it’s about how fast you can react to it.
John Allspaw (previously CTO at Etsy) has written about this: https://www.kitchensoap.com/2010/11/07/mttr-mtbf-for-most-ty...
It's worrying because it feels like a loss of control. But there must be control. And this what responsibility is. You should worry only about people who don't understand responsibility, not AI-inspired ones
I was under the impression that anyone that uses the MTTR abbreviation knows enough to understand that you need to balance it with change failure rate, deploy frequency, and lead time.
I call them True Believers
his worry is similar with search engine, I believe 90% of population don't even know how to properly do a good search in Google, that's why the info asymmetry still exists and the gap is bigger. It's just now we have AI.
I'm in a company going through this. Everyone outsources their thinking to LLMs and the results are painfully mediocre. The smart ones will use it to get their bearings on the topic then go to primary sources, the not so bright just ctrl-c ctrl-v.
Have you ever been in an HN thread where you're an SME on the thread topic and just been horrified by the confidently incorrect nonsense 90% of the thread is throwing around? Welcome to the training set motherfuckers.
LLMs do the same thing for what should be obvious reasons. If you search things that have some depth and you know the answer you'll be flooded by how often the models will just vomit confident half truths and misrepresented facts. They're better than they used to be, not just lying whole cloth most of the time, but truth is an asymptotic thing, not an exponential one.
Good point but he didn't go far enough. I would expand the AI psychosis to include all local optimization based on phony measurements , even time spent , DAU etc (which are mostly bots & synth accounts). In other words AI psychosis has been going on for 20+ years.
The only reason it worked has been expansive money policy and a larger share of the cost of goods being dumped into marketing value while manufacturing costs dropped abroad. so no one bothered to check.
I use ai to build a startup but I still decide what to build. Letting ai makes product decisions is where companies loose it.
The real AI psychosis is the expectation of 5x/10x productivity gains akin to the mythical 10x developer during the 2010s JS growth period.
At the end of the day, we can only read so much and take on so much work before we bottleneck ourselves. Cognitive overload leads to burnout. Rumplestiltskin vibes with this AI stuff…
Can someone please remind and refresh my memory what this whole debate was with what arguments?
Building things not to fail vs what Netflix does, build things to recover from failure.
Reminds me of this horrifying documentary: https://www.netflix.com/us/title/81095095
The groundwork for that was laid long ago with the idea of constant updates. It's been fine for years to ship bugs and rely on a rapid release cycle and constant pressure on users to upgrade everything all the time. To roll that back requires a lot more than toning down AI psychosis; it requires going back to a go-slow mindset where you actually don't release things until they're ready. It still needs to be done, but it's harder than just laying off the AI kool-aid.
Honest comment: it is transition time. This time is to make bets and take positions. Your humble position maybe.
I already took a couple of decisions. It will go wrong or well. But is was decided a year and a bit ago.
If you think the future will be different, stop doing the same you used to do the same way you used to do it.
My analysis is that the labour market will increasingly bargain salaries and will make pressure on you. So how safe is that compared to before? Maybe working for someone as an employed full time person is not the best thing you can do anymore.
The massive, destabilizing layoffs feel like AI psychosis to me.
What is described in the tweet may be worrying or not but it does not describe anything close to psychotic behavior.
I think the use of the word here is meant to invoke the vision of someone under heavy delusions or hallucinations, such as (what Hashimoto percieves as) the delusion that shipping more bugs is fine if AI can resolve them faster. To what extent this counts as delusion (and thereby psychosis) would depend on how deeply you believe that this and related opinions are wrong.
Im not afraid to say AI model trained on petabytes of data is better than me in many things.
Thankfully most of those things are a very small percent of my overall work.
If its a big percent of your work -> you are in trouble friend.
I was thinking about a different topic that could have the same headline just the other day.
Never mind code, what happens when the CEOs, or the investors, listen to the sycophantic voices of their LLMs?
I think it looks like every product becomes the next Juicero of its field.
You're speaking of my company and I'm forever grateful.
I'm afraid to say this out loud internally because I'm afraid of the next round of layoffs and I want to keep my job. So I just keep on shipping at a high pace, building massive cognitive debt and hoping the agents will get so good in near future, that there won't be the need for understanding the codebase.
Agents might get better. But who will own the code and take responsibility for it? The AI agent? The company who created the AI agent?
If e.g. a car crashes and does not deploy its airbags because the AI agent made a mistake in the airbag code, will the manufacturer be able to shift the blame to OpenAI or Anthropic?
I do not think so.
And therefore I believe that no matter how good the AI agents will ever become, the ultimate responsibility for the code will always remain with the companies that create the code. Regardless of which AI tools they use.
I see no other way to bear that responsibility by the company than to have people internally who will be responsible. And those people, if they actually want to own that responsibility, would need to understand that code themselves, in my opinion. Because relying on a non-deterministic AI agent's vetting is fundamentally unreliable, in my opinion.
The developers signing off on this will be "Human crumple zones" to protect the company from liability. Be very cautious if asked to sign off on anything like this.
This is why nearly all people that write code are not engineers, no "Software Engineer" would be willing to sign off on their code like this, yet this is level of safety guarantees real engineering is about.
Recently I had a request come through to allow finance analysts to vibe code their apps. During a discussion one of the finance managers let the cat out of the bag. Turns out our CFO had met fellow CFOs at a get together. They talked about how each of them were using AI. Our CFO was lagging behind and felt that we need to "accelerate" our usage of AI. He wants to push it just because he lost a bragging contest.
I call this Dinner Driven Development. That feeling of being Patrick Bateman when everyone is sharing their calling cards must be every C-suite's nightmare.
Now let’s see Paul Allen’s AI adoption strategy.
Oh god, he even has a dashboard.
Things like this make me realize the software engineering 'industry' is not a real industry.
There are people who write important software that the world runs on, but they do it outside the 'industry'.
A real industry should be responsive to events of nature, or at least the market, not vibes.
Oh its well within industry norms for leaders to make decisions based on dick measuring contests.
I don't know why you think a "real" industry would work in the most idealized way. The media heavily reports on the stupid insane crap of the tech industry, that doesn't mean every other industry is sane they're just not as vocal on Twitter.
Having worked in electronics, mechanics and software engineering the latter is definitely the insane clown show of the three. With the others sure you have some craziness once in a while but you are still being constrained by the real world and solid engineering principles.
Market is vibes! The price of something at a moment is, for example, what market participants collectively agree what the price of it should be.
Hate to break it to ya, but this is how most C-suites operate. Their job isn't to run a company well. It's to appear to the board/investors that they're running a good company.
It is a better play to do the popular thing in a way that measures as "ahead". Then it's hard to argue against a raise. But if you stick your neck out on your thoughtful expertise, it can take years or more for the value to come thru. You can easily be replaced by then.
The only antidote is a board that has a real working nuanced understanding of the entire industry. But this rarely happens, for many reasons.
It is really surprising how many of people that are running entire companies are ignorant businessmen
It is a tale as old as time (in the industry)
I wrote a while back, Most of the executives I have met really have no clue. They just go with what is being promoted in the space because it offers a safety net. Look, we are "not behind the curve!". We are innovating along with the rest of the industry.
https://news.ycombinator.com/item?id=46670190
"What, you guys are still not using Big Data? What's wrong?"
That is an uncharitable interpretation, IMO.
The CFO heard of a novel technique used by his peers in other companies, and they reported good results. He wants to try it within his organization too. As an executive, he is paid to (among other things) keep abreast of such developments in the industry and ensure that the organization he is leading is not caught flat footed in the market.
Peer-pressure driven management style. Fantastic.
I am guessing you were being snarky. Care to explain what about it you find about it objectionable?
Right right, I'd prefer management lock themselves in a closet and not engage with the outside world.
I’m at a FAANG and we have $300/day token quota. Personally I don’t use that much of it but management is pushing really hard for it. “the quota has been raised for a reason, use it”. Any task: “have you tried working on it with Claude?”. Every meeting “now engineer x and y will show you what he did with AI”.
It’s not all useless but most of the days I think I would be more productive if some processes were streamlined rather than if I had to throw tokens at them and still fail.
Of all the showcases I’ve seen the best are the ones written by people assuming that the token bonanza will not last so they used AI to build tools they wished they had. AI used to build the tool but by no means used by the tool, so if/when token quota gets reduced we still have a functional tool.
Innovation signalling.
Also at a FAANG here. Surprised you don't manage to use $300 in a whole day. It's almost trivial to productively use that much in under an hour.
Leadership is not being dumb, at least on this topic. If your token usage is that low, you just aren't using AI that much (even if you think you are.)
How are you able to get to $300/hr productively? (I’m assuming this isn’t fast mode tax).
not hard, massive elo stuff. every decision point needs to think up and implement 25 ideas and then rank them.
I use $30 a day to produce a decent amount of code. Certainly more than we need - thinking about/designing the correct solution/distilling requirements is still the bottleneck. How can you possibly even review $300/day worth of output?
It doesn’t have to be $300/day worth of output tokens. It could be like $290/day worth of input tokens to teach both you and the model about the problem you are solving and then $10/day worth of output tokens.
And what about you knowing the problem and the solution, but are just worrying about the impact downstream. Most of my time is spent managing those. I know the exact code to be published. And some time I already have it committed in my local branch. Then you need to make everyone aware of what it entails and that's usually how you can spend days on a simple bug or a change request.
Software is a big graph of interlocked rules. And if you can grasp the whole or the part you own (and you should be able to), it's often very easy to see the control points. You don't have a coding bottleneck anymore, you have a communication bottleneck[0]. Which is an organizational issue, not anything relevant to engineering.
[0]: See Naur's Programming as Theory Building and Brooke's Mythical Man Month.
If you give it $290 of input tokens for $10 of output tokens, you are doing something wrong. I.e. you paste the whole CI output into the prompt instead of giving it a link to the file, and then the AI greps its way through it (using a fraction of the tokens).
Sometimes AI overdoes things and it re-runs the whole testsuite because the tail command didn't have enough lines, but the other way round messes up the context so much so that in the end all that context is useless.
It could be thinking tokens or tokens passed in via RAG.
If you build your own reviewer layer/tool it will burn a ton of tokens. Millions of tokens of input.
You, review bots and first pass bots can chew through tokens. Also if you haven't put effort into your harnesses, the agent will have to spend more time and tokens figuring things out again and again
Use expensive models at high effort
I used Claude about a week ago to do a pretty intensive refactoring. Cleanup, initial modularisation, beginnings of a test suite, and better isolated build. In a span of couple of hours, and over a sequence of 20+ new commits, I burned a hair over $100 in tokens.
If you are working on a seriously large legacy code base, I can see how you'd get to >$250 on a bad day.
If you are burning through $2400 a day, you’re just wasting tokens on idiotic tasks.
He's rewriting Bun from Rust to Python now.
Do you even realize how insane your comment is?
"If you aren't donating at least your salary's worth of company money to another company every day, are you even working?"
Used to think exactly like you. That's why I know you all will "get it" eventually. Most companies and orgs are just so far behind the curve.
You might want to put this statement in your AI and ask if it was logically sound
Of course it's logically sound. The AI skepticism crowd is trying to tell me the reality I see before my very eyes and work with every day simply does not exist.
I know for sure that reality exists, and that they will either catch up or be left behind. Don't really need to explain myself beyond this.
What's more likely is that you are rationalising your religion. Some people break their conditioning others don't.
Believing something to be real that isn't is basically what psychosis means.
I do find it quite ironical that the thread is about the fact that Mitchell states that there are entire companies right now under AI psychosis and then we see solenoid's comments and they do seem to prove a few part of it.
Hackernews acts like a great litmus test indeed.
give some examples or real insights, otherwise it's difficult to take you seriously
This is a very common pattern with AI psychosis victims (and with crypto and NFT evangelists before). Comments whose haughtiness is matched only by their lack of content.
Its the same people. They didn't just up and vanish, they just moved over to AI!
"Used to think exactly like you until I accepted the love of Jesus, our Savior, in my heart."
No AI believer ever gives any concrete examples or evidence of what they’re doing with all the tokens and how it’s objectively helping them make the world a better place. Even for the shareholders (excluding the shareholders of Anthropic, or course), never mind the rest of us.
Why don't you explain it to us then? What are you actually doing with it? What type of products are you working on?
I've noticed that these comments (which are common) never seem to get a reply.. wonder why :)
Apparently both dev's and the AI are vulnerable to the Dunning Kruger effect.
Wouldn't they save an enormous amount of money by getting rid of either you and the token quota, or a bunch of other people to continue paying your salary plus this insane quota?
How? I struggle to use the 1000 Kiro tokens I get a month, and that only costs $20. And I use it more then anyone else on my team. Maybe we're just massively behind?
I am glad I am not on your team, the amount of slop they have to deal with coming from you must be overwhelming
300 an hour, that's insane
Not really, if you use the most expensive models and you have a large codebase stuffed into the context window
Also you regarding Claude usage limits:
So which is it $300/day is trivial to consume or $200/month is a completely reasonable limit, it can't be both.
You must be using a really bad harness or just writing very vague prompts. 20 Million tokens is a lot.
Seems like people are spending more time building tools than doing actual work. Lots of overlap too
In all fairness, doing actual work in this current slice of time is not what componies are prioritizing as of now.
That’s funny I’ve been doing that too
Trying to crank out all the tools I never had time to build because I think we’re going to get cut off eventually
May I ask what tools did you make so far? And what is on your roadmap?
Not OP, but I've been focusing on linting and automation.
Custom lint rules to encode best practices that previously relied on astute/alert code reviewer to call attention to. This is handy not just for humans but it steers the bots too. Or turning on some existing rule that required a big cleanup/migration to be compliant with. Now I just throw an LLM at it, since they're often laborious but mechanical changes. Which is the sweet spot for an LLM.
Also automating everything I can. That annoying release process that everyone hates but wasn't quite long/arduous enough to justify the time before? It's now automated. GitHub workflows for all the things.
This kind of stuff will forever be useful, even if the bottom drops out and the bubble bursts. And none of it is reliant on AI to run
Not OP, but a very simple example: I use AI to review my work before opening a PR for my colleagues to review. I ask it to review the commits in my branch. Instead of consuming tokens just to instruct it how to use git operations and other tools to find the commits since the base commit, I asked AI to create a little bash script to make patch files commit1.patch, commit2.patch, commit3.patch, etc, for all the commits in my branch since the base commit. Now I just use this script to prepare the context of commits to review.
I feel like an imposter here, I’m definitely not using AI as much as it seems everyone is :( I can’t imagine using hundreds of dollars of tokens a day. But maybe this little tip for reviews might be helpful to someone.
I also find it useful for review, and sometimes I use multiple passes to review for different categories. Like security, performance and so on.
Claude already knows how to use git and jj, very well.
Not op, made a tool to convert Microsoft OneNote notes to Obsidian canvas and Markdown. First it used a python lib which was too limiting. Then it used windows API to plug into OneNote and read the doc in its original XML form. That made the conversion correct and fully featured.
Same. I've used it for debugging failed canary tests which required scripts and very specific knowledge on the canary platform that I wouldnt of ever spent time on.
I also have scripts to fetch specific database assets and forward them to slack channels so I can easily share them with a group rather than manually running a query and generating them.
I had a theory about improving a product. I asked it to build an offline simulation setup to try various implementations. The results were a bit fishy but i decided to give it a try and A/B testing is showing similar results.
And now im vibecoding a locally hosted dashboard. This one is less useful for anything specific, and more of a minor quality of life improvement, but its fun to just vibe code and see changes happen occasionally. Its not a critical thing.
I find it very useful for debugging tasks like that but it always ends up costing me like $3 despite doing incredible work. And then one of the other engineers at my company will rack up like $200 in tokens in one day producing tens of thousands of SLOC and we end up actually shipping about the same stuff. Sometimes I wonder if it's bad agent use discipline (just pointing it at massive codebases and having it read it all from scratch each time) and sometimes I wonder if they're just using it for personal projects. Because none of that code seems to land in prod, and I've found that cranking out 10s of thousands of SLOCs at a time is a recipe for a mess.
But depending on how much you get paid hourly, $3 would be very little comparatively, no?
Yeah that's my point. You can get a ton of value for a few bucks so I'm not sure what these people are doing to torch hundreds of dollars. It's possible they haven't figured out patterns to make AI work on large codebases, and it's also possible they're just churning endless on massively bloated AI written codebases.
This seems seductive, but how do you get past the wall of "fixing XYZ or adding convenience ABC isn't on our pre-planned roadmap" so you can't get buy in from people who have to sign-off or deploy stuff?
Maybe that type of awkwardness is specific to my firm, but that's sort of what killed my drive to try to do that. We used to have one day every second week for that sort of work, but since it was scattered around, the tasks ended up disappearing-- nobody reviewed them and they didn't get merged.
So now they're trying to do a week-long internal hackathon to recover that vision, but I feel like that's going to produce a handful of big-bang ideas and not the 25 tiny tools that would actually streamline things.
I don't think we will. I think this level of token cost/availability will trend cheaper and faster, long term. These companies that spent too big and too fast might try to limit it and raise the prices and they might be temporarily successful but they'll very quickly be taken over if they keep doing it.
300 a day?? 7K dollars a month? No wonder they need to lay people off!
Well, presumably (hopefully) they aren't expected to work weekends.
No days off for the agents.
Yes, the cost of AI is a big contributing factor.
At Nvidia, we have no limit for Anthropic or Open AI models (for now) and are heavily encouraged to use them as much as possible.
Please don’t tell me you’re writing RTL
I'm not, I work higher level products. I've talked to a few people who do but I don't recall if they have different standards.
The fact that they've started promoting using the Caveman mode tells me that the unlimited usage policy is taking its toll.
What is Caveman mode?
https://news.ycombinator.com/item?id=47639077
Fwiw, nobody has ever suggested to me that I employ token compression in my daily workflow. I don't pay full attention in all the AI workflow demos I'm supposed to attend, but I don't recall that even being discussed. Is this an Nvidia blog or tweet you're referencing? I'm actually interested to see what they have to say.
He clearly works at Apple, and they aren't laying people off.
I'm not aware of a limit in my current role. There is, however, a leaderboard.
I by myself use now more than 15 accounts combined of all providers + API as well for external providers, more than 50K$ equivalent a month in API tokens, my team is doing the same thing, it's not really that much once you figured out the real automation loops and workflows, solving 300 issues a day with guarantees is common.
I feel that a lot of users are still stuck on Claude code or tools like this and don't really have a real argument about why they are even following the thread at all, everything has to be async for serious automation, you shouldn't even be seeing what Claude or any other model is replying (everything has to be digested with another model to increase relevancy and accuracy of the message so you can read faster (like a bot)), it's irrelevant, only human in the loop when a decision must be made, the rest has to be loops with all model, typical e2e, regression, computer use test, video into frames into all model loop and so-on.
That's interesting. What is the input into the process? Don't you need a PRD or a requirement doc to start with?
The unsubsidised costs can't be revealed soon enough.
Are companies using per-token billing? Why - is there some reason they can’t buy the $200/mo Claude plan for every employee?
Compliance
I'm pretty sure with AI there is nothing that complies to anything.
Staring with the fact that the whole industry is based on copyright infringement.
You're welcome to have opinions on that, but the answer to the person's question is objectively compliance. A corp can't get enterprise features like ZDR without switching to token based billing. That's why they aren't using subs.
This isn't some kind of new thing. There's always been an enterprise tax, like SSO.
The $200/mo Claude plan is not available for every employee. You can buy the $100/mo plan for up to 150 people, and then you have to switch to API billing.
Max 20x is for individuals only. (could probably have emps get it themselves, and reimburse)
IF they do individual billing the business doesn't get token reporting
They can’t track token use this way. Also it’s a massive violation of the model providers TOS.
Huh? I believe it’s completely fine for a company to pay for regular Claude subscriptions for employees, as long as they don’t share logins.
Not fine as per the ToS.
I can't find anything in the ToS that it would go against. I even asked Claude to check its own ToS and tell me if it's okay.
This is probably the fact that Anthropic's website serves different ToS based on your source IP. European ToS for the subscriptions forbids commercial use.
Yes, token use can be tracked the same way, just have to MITM everything. The ToS is a non-issue as it's not a legal issue, unless you plan to do business with Anthropic, not really an issue as you can always go to API later-on, in which case, Anthropic can't supposedly "ban you" as they are saying they don't record prompts.
Most startups do this (multiple accounts per employee).
Those plans are going the way of the dinosaur, ai provider loses money on them. Most enterprise offerings are already there, Anthropic changed theirs to $20/seat plus token usage a couple weeks back
I’m curious what FAANG is actually doing per-token billing? I’m guessing not google or amazon (since my wife and I aren’t aware of that).
"AI used to build the tool but by no means used by the tool" is a really good way to put it. Feels like the smart play right now is treating these credits as temporary subsidy and building stuff that still works when the bill comes due.
It is fairly easy to tokenmax by having and inefficient automation set up.
Not something I would do personally. But it is surprisingly easy to set up a claw that eats half of your token budget in a meaningless "research" task. Set it up as a cron job and you will soon be promoted for being an AI visionary
Unless other FAANG have the exact amount this is going to be Apple.
And no wonder why the quality of Apple software has gone downhill.
Apple in software development and design used to be very conservative. BSD like. Especially the lower end of the stack.
Now it is no different to other Silicon Valley companies.
How do you even use that much daily?
I have an unrelated question, please. I am trying to make a post and get this error: "Sorry, your account isn't able to submit this site.", you know why or have a solution for it?
Up to 80% of software projects fail. Most startups will fail. VC's and bankers know this.
Does using AI increase or lower that failure rate?
Does seeing a project that uses AI fail mean it wasn't going to fail if it didn't use AI?
To try to answer it with my gut: I imagine that we could see more projects failing, but the percentage that fail would be the same. Most projects that use AI will fail because most projects generally will fail, but the time and cost to get a successful project will lower.
The longer I look at the AI transformation, the more it seems like a people problem than a technology problem. The technology is undeniably there. The people are all over the place.
I am watching a 10 person company try to run 3 different AI initiatives in parallel. Everyone wants to be "the guy" on this one. I cannot imagine there will ever be a bigger opportunity to ego trip as a technology person. This is it. This is the last call before it's all over. There are many businesses out there that are beyond traumatized by human developers taking them on bad rides. The microsecond they think this stuff will work they are going to fire everyone.
The psychosis comes from the tension here. We effectively have The Empire vs the rebel alliance now. I know how the movies go, but in real life I think I'd rather be working on the Death Star than anywhere else.
My biggest grief, among many, is that the field is just no longer enjoyable to work in.
I cannot deny the impact of AI for my daily tasks at this point.
But I just don't enjoy the field anymore. With increased productivity, also coming from my stellar coworkers, it feels like we're rat racing who outputs more.
The quality is good, and having very strong rails at language and implementation level, strong hygiene, etc helps tremendously.
But reality is that the pace of product vastly outpaces the pace at which I can absorb it's changes (I'm also in a very complex business logic field), and the same might be true about my understanding of the systems which are changing too fast for me to keep up.
I feel mentally fatigued from a long time, I don't enjoy coding no more bar the occasional relaxing personal project where I can spend the time I want without pressures on architectural or implementation details.
I'm increasingly thinking of changing field, this one is dying right under our eyes.
I often read comments about HN users still delving at their place with technical details or rewriting AI code to their liking.
I'm increasingly sure that these people live in happy bubbles where this luxury still exists. But this methodology of work is disappearing across the industry, team by team.
Of course SE will not disappear over night, but the productivity expectations, the complexity ballooning are raising the bar where only incredibly skilled and productive engineers will be still able to practice SE properly, and as long as they meet stakeholders expectations or keep living in those bubbles.
I felt this way before LLMs hit the scene.
I'm trying so hard to pivot away because of this.
So rewriting gets cheaper and cheaper. New features fall more or less into the same category. Refinement doesn't.
The question is: Will we live in the world of breathless re-implementation, new features every week, rebranding every quarter or will we eventually discover the value of stability, software that does its thing more or less optimally for decades?
Recent examples of things like curl or Firefox are interesting in that regard. Will we end up with a nearly perfect HTTP user agent and stick with it for decades?
Preferring "boring software" over the shiny new thing is common wisdom.
Sounds like we prefer stability for stuff we use but not for stuff we sell.
The DevOps team at my company wants to hire a replacement for a very talented engineer. They’ve been interviewing candidates. The board got wind of it and someone not in their team decided they needed an AI Engineer, which is absolutely not what they want. So to release the funds they have been forced to change the job description and go after a different type of role altogether. It’s complete nonsense.
I don't entirely know what rational discussions that can not be had?
It seems like he is pointing out that Ai will increase the complexity of a system oblivion, and that this is the discussion that can not be had.
Bit I am more than happy to talk about how I am using Ai to reduce complexity and remove architectural debt that I otherwise could not justify spending time on.
So it sounds like he’s not talking about you. He’s talking about people who actively choose to ignore complexity risk and refuse to have a rational discussion about it because they believe AI will always be able to fix it.
The primary issue here is that CEOs and investors are particularly vulnerable to AI psychosis which is then forcibly propagated to the rest of the organization. Understandably, the perceived benefits are almost impossible to ignore, compounded by the FOMO of the AI first/AI native narrative being sold by AI influencers.
I don't think this is actually anything new. In large-enough companies, even before AI, it was and is quite common for executives to lose touch with base reality. I don't think anyone is under any delusion that people like Mark Zuckerberg intimately know the entirety of their corporate codebases. Everything is filtered through layers and layers of middle management whose summaries, cherry-picked statistics, and perpetually up-and-to-the-right graphs make it difficult to have an objectively informed opinion. Companies would, are, and will have mass layoffs that unintentionally (or, intentionally but with indifference to the consequences) fire key engineers whose loss results in "familiarity debt" within the systems those engineers owned.
Calling this "psychosis" is maybe a neologism but it's apt in perspective.
All that's actually new with "AI psychosis" is an acceleration of that phenomenon. The agents will summarize status faster than any middle manager. Claude will happily draw you any "up-and-to-the-right" graph you please, with the most common contemporary examples being "tokens burned" and "lines of code written". And vibe coding doesn't even require paying the cost of a mass layoff to get the "familiarity debt".
There have always been both good and bad engineering leaders. No tool will magically make a bad leader into a good leader overnight. There is nothing new under the sun.
Possibly psychosis. Possibly just serious ignorance and mob mentality. Leadership is supposed to be phlegmatic and measured; instead, we are saddled with hysterical hotheads. (Of course, when they are phlegmatic and chasing fads, then it does indeed resemble psychosis.)
Worth also noting is that while there is plenty to criticize about AI use — especially any cultish behavior surrounding it — plenty of naïveté about the quality of its results, there is a also a strain of categorical opposition to it among some tech people that is equally off and that has all the hallmarks of the chickens coming home to roost.
For years, many in tech gladly “automated away” all sorts of jobs. Large salaries were showered on them for doing so, or at least promising to do so (there was and is plenty of bullshit here, too). Now, AI appears to threaten to derail the tech gravy train, especially for SWE work that’s run-of-the-mill (which is most of it). Now automation is bad. It’s a delicious juxtaposition.
AI psychosis is real, but at worst is only premature. AI-denial psychosis is far more pervasive, and will bite far more people in the long run.
I find talking about X psychosis (or generally using mental illness metaphors) unproductive. It sets up the conversation to be "nothing else to do with this person".
Maybe the problem is you, but you won't figure that out if you think the other person has psychosis.
For example, maybe you need to do a better job explaining, changing your language, simplifying things, being more concrete with consequences.
Or maybe you aren't understanding that the other person has different objectives/ loss function that makes them make seemingly weird conclusions.
How about AI Derangement Syndrome?
Sometimes I feel like "doing it with AI" is the new "rewriting python in rust".
Rewriting in rust does makes things faster but if an algorithm is O(n²), the improvement won't take us much farther.
Similarly with AI, if complexity is not structurally adressed, the velocity gains are but temporary.
He is a billionaire and still thinks at a developer level is pretty remarkable! Hope other billionaires pay attention to this!
I'm just waiting for my current company to have a Sev 1 CritSit so I can document the bejesus out of the root cause and expose our non-technical AI evangelist leadership as the sort of goons most of the senior development staff already suspect.
Only by walking us into some revenue or customer impacting failure - through inappropriately having junior devs doing senior level things - will some sense of sanity start to prevail again.
Oh man, if only. The top brass driving this screaming frenzied MORE AI crusade will never face the firing line no matter what happens. It will either be a) "mistakes were made" and nobody is really at fault because we're all trying to change the world or fellate the future or whatever the line is, or b) James, Sam, Jesse, and the rest of Team B (none of whom are truly top brass) are getting fired out of a cannon into the sun as a warning to the rest of the plebs.
Company I just left is reportedly now using Claude to analyse the metadata generated from the company MDM that tracks actual laptop use, and then pulling people up if they're not working "enough".
They're also reportedly now giving staff AI-related "homework" in an attempt to force staff to use AI more.
Interesting. Were those Windows or MacOS laptops?
Both
Corporate management in the USA is focused on the short term and reactionary.
Changing this focus is not easy but one thing that will usually do the trick is economic issues.
In other words; in order to get any serious consideration, something has to be broken.
AI is perfectly capable of doing this given enough time.
At work they are purging any developers who are not all in on AI. I must constantly be in full support of AI to not get fired, despite whatever my true thoughts are, including anything I post on LinkedIn. There can be no doubt.
People just need to calm down. We're scaring the shit out of ourselves for no reason. Just like, chill man.
It's a tool; not the second coming.
It has potential to remodel society though.
Cars replaced horses.
But AI is poised to replace a large chunk of brain labour.
Where's the ceiling?
A stable civilization that doesn't devalue human life and well-being to the point of absurdity.
thats the ideal ceiling, but I don't see why we won't surpass that.
I think people need to make sure ceilings are built, and we can calm down once we're done.
Its a tool that is built by people who are incentivized to have you addicted to it.
And how.
OAI has a billion users. They can train the model directly against the metric of how often those users use the model. (mutate it many times, test which one works best, keep that, repeat). The model would (did) learn to be sycophantic and persuade the user to keep talking with it by constantly suggesting new things. The hype only makes this easier for them because it leads to people believing that the model is super-credible and trustworthy. They do this because it is easy and it makes them money.
I think there's a reasonable argument that our entire society right now is under AI psychosis:
The stock market keeps going up in the face of the indefinite closure of Hormuz. We're investing in datacenters at a scale that only makes sense if AI capabilities continue to advance to the point where they surpass most humans at most white collar tasks, if not reach superintelligence.
And what are the possible outcomes?
- Bust. We've come away with a useful tool but the hundreds of billions of capital expenditure were thrown away on a pipe dream.
- Success! We're the dog that's caught the car. Then what? Currently the political debate is, to caricature only slightly, between "oh no the datacenters will use more water than golf courses" and "lol what are you going to do, regulate matrix multiplication?". How the hell are we going to cope with introducing a new intelligent species?
Either way, it sure seems like we're collectively operating more in the interests of the future AI than in the interests of humanity. What is this, if not a sort of psychosis?
Another possibility is that the hype continues, growing and growing and sucking up more and more resources, and the piper has to wait yet another day to be paid, until someone figures out how to pivot to the next big thing and all the debt (financial, social, environmental) gets carried forward and we keep going.
In other words, BAU for the last few thousand years.
Paypal Mafia -> Crypto Mafia -> AI/LLM Mafia -> I'm calling Biotech/BCI Mafia, WW3 Military Industrial Mafia, or Energy Mafia next.
Water Mafia
Nestle?
The Resnicks: https://perfectunion.us/how-this-billionaire-couple-stole-ca....
Let's remember that these people have names and addresses.
For what purpose? Murder? Arson? It's amazing how often people say things like "no one is above the law" whenever it's convenient, then totally flip the script when it's not.
why is it that you give a pass to the violence and death in the dozens, hundreds, thousands and millions at the hands of billionaires who regularly kill for profit...
yet balk at someone deciding to fight back in kind and on an exponentially smaller scale, comparatively speaking?
Because the propaganda trained us well. A single loss is a tragedy, a loss of thousands is a statistic.
It's just good to remember they're not some kind of magical deities, but regular people like you and me.
IIRC they're already in the Military Industrial Complex with companies like Palantir, Anduril, and others.
Good point. I guess I feel they're still getting into position there, and haven't really had their opportunity to blossom yet. The average western citizen's experience of war is still just slightly increased food and fuel prices.
I don't think the majority of humanity will ever accept "AI" as being anything more than a fancy computer, let alone a 'new species', even if it was proven sentient.
Nobody has to believe AI is alive for governments, markets, infrastructure, labor, and energy policy to start orbiting it anyway.
If this starts to happen then heads must roll and data centers must burn
Otherwise humanity is over
Well, I have bad news for you.
People will anthropomorphize rocks.
You just need to embody the AI in something that moves, and then people will definitely treat it as a new species. Already happening in my town with delivery robots: when they get stuck on a kerb, a person will stop and help them up the kerb while saying soothing words like to a pet: “There you go, little guy, now everything is alright.”
Why wouldn’t it? The closure leads to price increases which leads to inflation which leads to non-dollar assets (ie stocks going up in value)
Second from a US perspective the strait matters the least it has since world war 2. If the price stays high a bunch of fracking will come back online.
Because uncontrolled inflation will destroy your economy
yes, but that's future us's problem
Another strong contender for humanity's epitaph
It is more than one fiscal quarter away. May as well be a lifetime.
I think this argument proves too much. Historically energy shocks have led to recessions, and in recessions the stock market usually doesn't go up. And the US economy is certainly exposed to global recession regardless of whether we're a net exporter of fossil fuels.
Well, there are quite a number of factors. I think you're right that "it's inflation" is a little too simple, but it does seem to be at least a significant factor, in my opinion.
The Strait of Hormuz is, basically not a big deal unless you're driving your big ole' truck. Americans are price sensitive and so some companies will have to absorb pricing increases, customers will absorb some others, and so forth. In other words, business as usual. Of course the closure of the Strait is a big problem for most of the rest of the world. They better get on with figuring out how to get Iran to stop being so chaotic in the region or we'll just keep it shut down indefinitely. No big deal.
Because the United States has so many advantages (primary global reserve currency, robust and efficient capital markets, highly sophisticated and dynamic economy across all sectors except luxury goods, &c.) it's able to weather these storms much easier than most other countries. As a country that also imports so much, if we spend less on imported products that's less of a problem than not being able to sell products. A recession isn't great, but the current parameters seem to suggest to me it's less of a problem for the United States - perhaps why we're in part seeing stock market valuations continue to climb.
Are you serious? Even ignoring the other things that ship through there, a significant disruption to global energy supply is significant to most people. If you're not driving a truck, you're probably using goods that contain plastic or took energy to produce or were moved from one place to another in fuel-powered vehicles. If, somehow, you're not, you're probably using services that are.
Not to mention the countries heavily reliant on LNG from Qatar that are facing a very difficult time.
The best course of action now is to spend less time criticizing the United States and more time working with the United States, sending assets, military capabilities (if able), or at least providing political and diplomatic support &c. to stop the Iranians.
The world let this disease (IRGC) fester in the region for too long, and now because of that the fix is going to require significant pain. The IRGC in its current form has run its course and will not be allowed to threaten American interests, allied interests (whether that's Israel, UAE, Saudi Arabia, or otherwise), and they will not be permitted to build a nuclear weapon or threaten global trade.
Hate to break it to you, but the IRGC isn't going anywhere.
The reason nobody was dumb enough to attack them before is that it's an unwinnable conflict. They don't need a lot to close the Strait of Hormuz, a few guys rolling mines off a beach would do that. And they have a lot more, like missiles and drones to do damage at a distance too.
And it's a regime that has at least a million loyal fanatics ready to fight for it (the Basij, the org that did unarmed meat waves against Iraq to defend the regime). So any invasion is an absurd proposition.
So what, the hope is that the theocratic kleptocracy will give up? Not even a child could be so naive. They literally believe in martyrdom, whacking a few of the top dogs means nothing.
It's like the Kims, nobody can unseat them. Only this is far worse, because Iran has the leverage of Hormuz, and it knows it can wait - because they don't care about the people - while the US and global economy suffer until they fold. Especially with midterm elections coming, the US will fold.
The festering disease is the United States of America, not the IRGC.
So the best thing the rest of the world could do is send their own people to die because the US keeps bashing its head against a wall here since the 50s?
What's your sales pitch exactly for how that's the best thing for the non-US rest-of-the-world? What's the US's post-WWII track record, success-wise, in regime-change foreign wars, how much would you trust the US on this one?
Countries should be sanctioning the warmongers that caused this. Confiscating trumps golf courses would be a start.
Yes I'm rather serious. For the US it's not a big deal (again unless you're driving one of those giant trucks where you're spending $300 to fill up, not my problem). Are $5 gas prices great? Not really, but is it a catastrophe? No, far from it. We have dealt with high gas prices before and we'll see high gas prices again. We just learn to live with it and find other ways to get efficient or whatever we need to do.
Some Americans need to have their understanding of the world checked. If you think high gas prices are the end of the world, just wait until we have a real problem. Are we going to be incapable of fighting a war because Netflix and Pepsi prices went up or it's too expensive to coal roll down the highway?
Separately as someone who supports both Ukraine and the US and taking down the Iranians it's amusing to see each political tribe get mad about gas prices as it is convenient for them. When Russia invaded Ukraine, MAGA was screaming from the rooftops and putting Joe Biden "I did that" stickers on gas pumps. Now that we're taking on the Iranians all of the commies are doing the same thing (aren't gas prices good anyway since we need to do something about global warming?). Neither side of populist is worthy of serious consideration. Stay the course, whether that's supporting high gas prices because of Russia or because of Iran.
The US does not exist in a vacuum. Cuba just ran out of fuel. These things have cascading effects. Even if you do believe the U.S. exists in a vacuum and you don’t drive a big truck, there are still obvious effects. Spirit airlines went bankrupt, for one. Will this be a global catastrophe? I hope not, but it could be if we’re unlucky.
It won’t be a global catastrophe but if folks around the world think it will be they better figure out how to stop the IRGC and get the Iranians to knock it off. Otherwise we will just keep the Strait closed and deal with it. Don’t forget, Iran is the one making and selling drones for Putin to bomb innocent Ukrainians. That alone is a good enough reason to bomb their military capabilities.
Cuba ran out of fuel because we took out their thug partner in Maduro. If they wanted to drop the whole authoritarian communist dictatorship stuff and their involvement in the disaster that became Venezuela and partnering with the Russians then they'll be better off.
Do you like being able to buy food?
Yes, and will continue to do so even if it’s at higher prices. What in the world do you think is going on here? Do you think the US is going to run out of food or something because prices are a little higher?
You're right, ever since we developed trucks, trains, and ships that run on pure atmospheric air, we haven't had to worry about pesky price fluctuations on every physical object that we buy or sell!
Worst take I’ve ever seen on this website.
No. Not all goods/services have the same price elasticity. At some point, people stop buying some goods if they are too expensive. They stop commuting to work. We start to see breakdown of the supply chain.
Literally 100% of many towns in the US depend on trucks to deliver food to their grocery stores and the inventory on hand usually only lasts a few days. Once those trucking deliveries become unaffordable for either party in the contract, society starts to. Real down.
Consumers don’t magically make more money when the price of gas rises. It starts to crowd out their ability to spend on other things. The poorest of the working class likely has to commute the furthest so they will end up sacrificing something to keep paying for the commute - food or rent or utilities.
The US doesn’t weather this because we have “a sophisticated supply chain”. _If_ we weather it, it’s because we created the US SPR after the last major oil crisis and we have significant domestic supply (although not all oil is fungible so we might not have enough light sweet to keep the economy running at 100%).
We are handling it just fine. Your perspective of struggle is very wealth-oriented. We aren’t struggling at all as a country.
The second problem with your argument is that you’re using it as an argument against the war but it’s actually an argument in favor of the war. Why is that? Because as Iran continues to load up on missiles and pursue a nuclear weapon they reach a point where they can assert control over the Strait and shut down shipping pending tribute to their theocracy (maybe if it was a Christian one you’d have a bigger problem with it? Idk?) and then we couldn’t do anything about it. The world isn’t static. Stop treating it as such.
Equally bad take in your response.
Iran never attempted to develop a nuclear weapon. Literally 30 years of Netanyahu threatening it is just “weeks away” and Trump was the only US president to get suckered into that argument.
They have always used the threat of developing it as power, both domestically and regionally. It was 100% the exact same thing with Sadaam Hussein and WMD — they want to appear to have the strong weapon, but simultaneously don’t want to develop it or use it (until, ironically, Trump / Israel took out their leadership). Also, we were assured their entire nuclear program was “obliterated” summer 2025, so which is it? It can’t simultaneously be non-existent and an urgent national security threat to the US.
Now that Trump started the war in Iran, we are playing chicken with people who don’t care if they die or not (or so we are told). Not exactly a good position to be in against the “world’s largest sponsor of state terror”. We kicked a hornet’s nest and our political leadership didn’t stop to ask the experts (military, political, or economic) what were the completely predictable second and third order problems with this strategy. We couldn’t even get Trump to wait long enough to fill up the US SPR before starting a massive war in the Middle East.
To reiterate, we haven’t felt any of the actual pain YET. My arguments thus far are only of current harm, not the likely harm in the near future. That’s for next month’s comments.
The shock is smaller, and oil’s importance is less so less likely to cause a recession. In the 70 price went up 400% and oil was rough 1.5x more important. Today price up 100% so the past oil shock was 6x larger
You are pretending like the oil crisis of 26 has already run its course. In the 70 crisis, we have the hindsight of several years of how it played out. We don’t even have 1 week of data after the last ship leaving the Strait docked in the US.
Also, the US SPR was created in 1975, so we are going to get to see if it actually works to absorb an oil shock like this.
Most likely there will be some places which are almost unaffected while others are going to see unaffordable price spikes (more than 400%). The pain won’t be spread evenly.
Success is also bust: Money becomes worthless. Everything you know and are is now obselete.
Furthermore, if you try to make your own decisions, you would be outcompeted by someone who has outsourced their brain. And, of course, since intelligence and labor would no longer be scarce resources that humans can use as leverage, gunning you down if you protest wouldn't really harm anyone in power.
People say that LLMs won't take us there. I think that's accurate, but there's a great deal of research going towards the next breakthroughs. How much are you willing to bet that all future attempts will fail?
We're trying very hard to build an ugly future.
I highly doubt that. Humans might become obsolete, but money will almost certainly still be a thing.
Interesting. Do you think a super intelligence would use our currency?
Yeah. I think the economy will be by AI for AI. Markets will still be the most efficient way to allocate resources, and AI will probably still want to achieve infinite growth of capitalism.
Resources are finite though
If we are successful building an "ultra" human AI, it will require massive amounts of energy. That translates directly to "money". There will always be money unless someone finds a way to negate the second law of thermodynamics.
But if it replaces people that make money who’s paying for it. None of this makes any sense long term and no one has a plan for it.
Weird that you mention the stock market and then conclude that there are only two outcomes: bust or success. If anyone can learn anything from the stock market it's that boom and bust are cycles that oscillate around a trend and everything tends to revert toward longer term trajectories. So, yes everyone is caught up with and overhyping AI and yes there will be a bust after the boom at some indeterminate point but that isn't the end of the story and we'll see a rise and further oscillation afterwards while we get better at applying the technology.
Yes and my partners father has been calling for a bust for the last decade and everything has simply taken off.
I don't see any taking off actually
I think in a way we have seen a take off, by detaching the hype bubble from success metrics. Companies making products being bought by companies making products, that are hyped by machines, bought bymachines and evaluated for viability by machines. If economy is to hack the API of VC this has taken off! For now.
I think they meant the stock market more generally? S&P 500 is up what, like 4x in the last 10 years?
If I would drink 4x more beer that would concerning, but the number going up 4x should make things better how exactly?
This was said in the context of a person predicting a stock market bust, so of course a stock market index price is the relevant number here.
Yeah, fair. I still can't shake off the nagging feeling most of it being a scam somehow and not the business as usual scam. The gut feeling that things proclaimed and observed don't add up.
That's just inflation, which is primarily controlled by the government via the money supply. It doesn't mean anything. What does mean something is the severe deflation in wages and consumer goods - why is all the money printing remaining in the rich person's realm instead of trickling down?
Peasants aren't picking up enough shit after the rich to distill traces of gold from it, that's why
Inflation is not up 4x in the last 10 years...
Depends what you measure. I already said it hasn't been affecting wages or consumer goods and that's weird.
As a bear that's been very confused by markets failing to exhibit mean reversion in 2019 and 2022, and now with the Hormuz energy crisis, I've thought a lot about this. There's a lot of new things happening. Fed/QE intervention that has never stopped, just been more well disguised. Fiscal/government spend intervention. I think Mike Green's work on the rise of passive investing is really good, in particular explaining how it prevents mean reversion in absence of changing net cash flows into passive instruments. Passive will also induce or worsen the bust if net cash ever starts to flow out passive. Green's youtube interviews are great.
All to say, your SO's dad would have been right at any point prior to the current financial cycle. Knowing what's changed doesn't make forecasting easier though.
I mean, there's a reason I started with a distancing phrase about a "reasonable argument"; I think what I'm suggesting is an interesting lens but does not capture the whole picture.
But also, even if bust is business as usual in the big picture and not a social disaster long term, it's of course not what individual investors want for their particular current investments.
You're describing the Gartner cycle: https://en.wikipedia.org/wiki/Gartner_hype_cycle
No, he’s describing the Business Cycle, with literature about it predating Gartner by a few centuries: https://en.wikipedia.org/wiki/Business_cycle
Your comment and the parent comment are not in disagreement.
Why wouldn't it? The value of the USD is decreasing, the value of the companies to the world stays the same => stock price in USD increases.
The real thing to analyze is "amount of VOO shares you need to buy a Chipotle meal / Uber ride / 1 month's rent in SF / etc."
If we achieve runaway AI, the stock market goes to infinity. So from an expected value standpoint, massive spending on it is worthwhile. Even if the odds are tiny, the payoff warrants a massive bet size.
Not really. If run away AI ever became a real thing, the stock market would be completely destroyed since most companies would become worthless.
But a few companies will skyrocket and completely offset those.
I can't imagine humanity, let alone any company or the stock market survive runaway AI.
You underestimate the non-computerized part of humanity. Even with automated plants building other automated plants the propagation speed will be highly constrained by natural factors, plain unavailability of resources, and xenophobic nature of humanity of course.
Companies and the stock market don't need humanity.
heh this is the trick. The tech companies will angle for a bailout and they'll benefit from all this speculative data center building. Compute is generally useful.
It’s useful for a while. Hardware has a pretty short useful lifespan. I’ll be curious how the landscape will look in 10 years as it comes time to replace all of these servers. Maybe we’ll extend the lifespan, or usage will continue to grow, or we start shutting down datacenters.
they're already all extending the lifespan for accounting purposes haha
IMO, what's happened is a few richest investors in the world had access to the uncensored tier of AI, talked to it and came out with impression that they've witnessed something so dark, so much beyond anything we can imagine, that the only course forward is towards the transcendent abyss. Call it AI psychosis or demonic inspiration, but they are the people who control the economy, so they are dragging everyone with them. "Operating in the interest of the future AI" is a neat way to put it.
People that don’t understand current AI likely have no idea how to differentiate Opus from some super intelligence. Further in their domain with the safeguards off it probably creates capabilities never imagined. To me they are making that leap of expecting continued capability improvement and their framing is “what I already saw is fundamentally game altering”. It doesn’t need to imply anything further, yet.
Here is a comprehensive, achievable plan to take over the world. Do not distribute outside of airdropped isolation state:
[plausible sounding nonsense]
Oh wow that gives the billionaire class so much intellectual credit they don't deserve. No, they see the same ChatGPT we all do and their mediocre brains with zero self understanding (see andreessen's explicit comments to this effect) determine "it's a new life form ! It's brilliant / conscious / my new girlfriend!"
Never overestimate the billionaire class....
If there is a psychosis, what is it? It is not an AI psychosis - modern AI started in the 1940s, or by some definitions before, and made progress up until 15 years ago to where deep neural networks became viable. And it has been progress on every front since then. No psychosis, it is doing well.
You mention the stock market, and that is another story. Cryptocurrencies, sub-prime loans, dot-com crash, Asian financial crisis. The economy has veered from crisis to crisis, overproduction and overproduction to crashes and bailouts.
AI is doing just fine - the past 15 years are a success for it we did not see in the decades before. If the economy as constituted is dealing with this in a "psychotic" fashion, it would not be the first time.
But the homemade god would liberate the elites from the nightmare of being responsible for running the planet into the ground. AI jesus take the wheel!
Are economists still trying to sell us the fairy tale that the market is rational?
The saying "the market is rational" simply means that assets are priced using all currently available information.
And then your bias makes you ignore all available information, so this doesn't mean anything.
Bias affects sentiment regarding future performance.
And it does mean things if you understand the meaning of the terms in the context of "finance".
You seem to fall into the same set of criticisms as everyone who’s bearish about ai. It’s somehow so powerful that we can’t handle the ramifications. Meanwhile, it’s a waste of money and doesn’t do anything. You have to pick a criticism and stick with it. Otherwise, it’s just angst-driven noise.
My car drives itself. That's a $18T global market.
Also $1T in data center investment makes sense when you realize that companies are racing to create virtual white collar workers. Google spends $9B a year on software engineers.
Which will take decades to become addressable. Self-driving cars work OK in a few cities in one country. Expanding that to be able to cover Mumbai and Omsk and Nairobi will require significantly more work.
Does it make sense? How much would the resulting virtual white collar worker cost? Because datacenters have running operational costs, and so do the people operating them and working on the software that runs in them.
This probably ends in a deflationary spiral. The ai replaces the jobs, the lack of jobs chills demand, the ai becomes cheaper because it exists in a commodity market.
The money printer will be used, and maybe it all works out - or we see wealth hyperinflation and build out our own aristocracy.
Market for who? Who is left working? If we can’t answer that question then we’re not prepared for what’s next.
Huh?
Hundreds of billions are changing hands globally, every week, at the retail level alone.
And that happens literally every week, week after week.
That constitutes a massive market in any sense I can think of.
That won't happen any more if nobody has jobs. It's also completely irrelevant because how did you just connect the self driving car market to the entirety of retail sales?
So you think a trend that is growing year after year, yes global retail sales grew from 2025 to 2026 week over week, indicates a future collapse?
That needs a way more complex explanation than simple gut feeling.
THIS is the question!
Henry Ford II: "Walter, how are you going to get those robots to pay your union dues?" Walter Reuther: "Henry, how are you going to get them to buy your cars?"
Ah, the elephant in the room. Nobody seems able to answer this point, or even talk about it. Occam’s razor sure doesn’t imply a good outcome.
For so long, people, especially politicians, have said that companies want to create jobs. But I think most companies want to create profit.
And for so long, I've had people tell me to just get a job. But I tell them that I don't want a job: I want money and I want something to do. Those two things don't have to be together.
I think this is the hard part: philosophically so many of us have learned we need jobs and don't realize a job can be decomposed into money and something to do.
So I think we need to start looking more creatively at 1) how people receive money from others and 2) how people give services to others.
You’re trying to create nuance where there is none. Creating jobs exactly means “I want to pay someone less than the value they bring in to my company” and this has been true since forever.
Nobody cares that you want money and you want something to do that you enjoy. Nobody ever will.
If you actually dig into all the social programs that exist at least in the US, they’re just a massive payday for a small group of people under the guise of bettering humanity.
College/education is a fantastic example. Education as it has been established today is a joke. The humanities were originally established for rich bored wives to have something to do. They were never meant to create value. Colleges hang anvils around the necks of naive children via loans telling them “yes if you major in history you’ll have a job!” This is a joke, and a bad one.
Huxley was on to something. If everyone is educated, nobody collects trash, or chops lumber, mines minerals and metals, etc. it’s a big fucking not-talked-about open secret.
Nobody cares, either you bring something to the table someone else can exploit for money, or you lean into “I’m helpless and the government owes it to me to take care of me because I’ve been indoctrinated into learned helplessness.”
“AI” will at best lead to anarchy at this point, if all the grand visions of the billionaires comes to fruition. People have already tried to kill sama and burn his house down. Wait until armed humvees are driving around data centers. It’s coming.
Well the essence of capitalism might be that people who own the capital receive money for owning it, not doing any labor on it necessarily.
So when we talk of people doing labor for money, we are assuming they can only own their body and receive money from that?
Ah, well, nobody needs to buy them if the robots just provision everything for the people who would have otherwise had their businesses make cars.
Driving a car is a chore, not a job (usually), much like washing dishes is. Dishwashers did not produce an economic collapse.
OTOH replacing people with AI would indeed bring about a huge economic downturn. What would be good is augmenting humans so that they can do 10x more. That would enable things that are hard to imagine exactly now, much like computers enabled interesting transformations in the society from 1980s to 2010s.
The current crop of AI is by construction unable to reach the human level of cognition, but it is quite good at doing some symbolic manipulation tasks. We will get used to that, and will integrate that in our workflows. Humans are still going to be needed.
Tractor-trailer trucking alone is the 13th largest profession in the US. It’s not unusual for driving to be a whole profession in and of itself.
Fair, but I spoke about cars, the commute / chore kind of work, not trucks, a commercial job.
And do you feel that the industry in general, and individual companies are currently trying to augment / 10x their workers and have everyone share in the 10x profits that will bring? Or are they jumping on opportunity to try and cut costs by even single digits, by replacing those workers with AI and it's not their problem what those people do from there?
If an employee brings in more profit than before, you want more such employees, not less.
You have to cut costs when the costs do not bring you enough profits.
That assumes the market is infinitely expandable.
If in fact you can meet the same market demand with fewer workers and the market does not expand accordingly, you get deflation and job losses.
That's not a new market, that's a new feature in an existing market. Lots going on in transportation and I'm not seeing any scenario where self-driving cars vastly increase total output vs just eat up other forms of transportation and change where people live/how long they commute.
Similarly, many companies are trying to be more efficient - "do what we already do, but better". That's different than growth.
What could Google do with 9B on software agents? Let's say the future of them is amazing and this means they could write 100x more code than they can today.
Has Google recently showed much ability to turn "more/faster code" into "superbly profitable new market"?
Someone's gonna have to crack the demand side issue for anything transformative to happen.
Well they are projected to spend $175 - $185B on capex in this year alone most of it for AI buildout. Lets say only 150B of that is for AI. If they can then somehow replace all their software engineers with AI that they then run for free and depreciate over 10 years then they just replaced 9B a year software expense with 15B a year depreciation expense for the next decade. Yes this is grossly oversimplified but it still illustrates how crazy high of a bet they're making on AI.
That's assuming they only use it to replace their software engineers and make no money selling AI usage or using it for anything else.
I believe the One Big Beautiful Bill Act allows full depreciation in the first year: https://www.bassets.net/blog/obbba-depreciation-2025-2026-gu...
That's a bookkeeping issue, it doesn't affect the argument at all (which is that the capex has a finite useful life over which it would need to pay for itself).
Yeah, just tangentially pointing out that asset depreciation rules in the U.S. changed recently. Could explain some of the crazy magnitude of this year's spending spree.
No. It doesn’t. And if you’re defining “drives” as “it drives as well as I do” then you probably shouldn’t be on the road.
Nothing about any of this makes sense. Tell me, when all white collar jobs are replaced by AI, where will the customers come from? Who will have income to afford your products or services? The poor barista whose surveillance videos are training the robot that will soon replace them?
Leaving aside any consideration of human compassion or questioning of the purpose of an economic system (hint: it’s not just an abstract machine), shrinking the pool of potential customers by orders of magnitude has never been a recipe for sustainable success (let alone growth).
It’s time for humanity to admit that this is the end of the line. We had a good run, some beautiful moments were created for shareholders along the way. Let’s take each other by the hand and walk into the darkness together. So Long, and Thanks for All the Fish
I don't know. We invented machines that can answer arbitrary questions and are quickly demonstrating the ability to answer questions no human has been able to answer. We're sending more rockets to space in a week than we have in the prior decade. My car can drive itself. We experienced a global pandemic and within six months engineered and scaled the mass production of a vaccine to mitigate it. We also just invented a weekly shot that nearly cures the most common cause of non-natural death. All of these things are new in the past ~five years. There is no definition you can invent that does not classify the times we are living in, right now, as the most impactful ever, in human history.
Such as?
5.5 Pro has been leveraged to solve at least two previously-unsolved Erdos problems [1]. Whether these were unsolved due to being seriously untried by humanity, or because of their difficulty, isn't relevant; no human proved able to answer them, while our synthetic intelligence systems did.
In my personal life, I have leveraged these systems to design code that I don't believe I would ever have been able to designed. And, because no other human may attempt to, this means the same thing: That no human would have been able to do it. Things like reverse-engineering niche APIs and digging into binary files to diagnose weird format conversion issues.
[1] https://x.com/DavidTurturean/status/2054942008817451195
That seems very relevant to my evaluation. I can pull out my calculator right now and solve a problem no human ever has.
True! But calculators are already priced in. LLMs now unlock a whole new class of problems that we can automate the solutions to; what we're seeing is the markets trying to figure out how to price that.
The people with capital are gambling that this will be an innovation good enough for the first player to take all. That’s where the hubris comes from. You’re a billionaire and you have the chance to rule the world if this plays out, and if you don’t, someone else is going to. That’s where all of this is coming from
If we want to understand a phenomenon, we should be careful with technical terms. It is not "psychosis" [1] anymore than bad software that makes mistakes is "hallucinating".
The truth is simpler and less dramatic: hapless ambition-monkeys who climb the corporate ladder are demonstrating that they are not promoted for mental acuity. Corporations, after all, do not serve "the interests of humanity" — they are an organised collusion system that diffuses responsibility and anonymises negligence. And when it works as intended, shareholders rejoice.
For companies that can afford to build data centres, the latter are seen as a sure bet that can't fail, like building a bridge or buying new computer/hardware now with the pile of cash on hand without necessarily knowing which OS/software they will install. They are even planning to restart or build nuclear reactors. [2]
[1] https://en.wiktionary.org/wiki/psychosis
[2] https://www.cnbc.com/2024/12/28/why-microsoft-amazon-google-...
The tone of the twitter post feels very personal, and emotional, and I am sorry for the author. I hope he can find peace and calm with the pace of change to put forward his best self without needing to act like he needs to defend or fight something.
The energy feels misdirected and maybe also a communication issue, I think spreading awareness needs to come not from attacking and also not from attempts to change people’s perception. It’s also quite challenging to distill a concept when it’s new, we learn both from our experiences and experiences of others; but, so far, these alleged systems that will eventually collapse, haven’t done so yet and it makes it sound like you’re preaching and predicting, condemning even, rather than raising awareness and education.
Not trying to sound hopelessly optimistic either, just that the other extreme isn’t also helpful, and that a spectrum is not what we want it to be but what the collective shapes it, so saying psychosis is rejecting the harsh reality that they’re far removed from your worldview and not working towards an understanding.
EDIT: Maybe I'm old and I don't get twitter, I also don't know much about the challenges he faced communicating his concerns, I sort of had a meta comment with the intent of "try listening more first, some people are difficult to reason with but respond better if you just let them speak and look for a teachable moment during the conversation". Anyways, I'm in agreement that there's too much unsupervised AI in the wild, I'm not saying he's wrong more like saying that doubling down on "stop doing that" will likely be ignored by those that are already ignorant to it, hence what I wrote above.
yes, the tone feels personal, and I feel happy for the author for expressing it on a platform that is desgned for it.
He is clear in pointing out the hard earned lessons we have learned before and how the current actions are essentially undermining it. This is dumb (i agree) and he expects better from people whom he respects.
it's clear, personal, logical. I don't understand what your criticism is.
It sounds like you know a lot more about him and the context than I do, the angle I am coming from is mass audience. This reached far, to the point I have no clue who he is and what else is going on. That’s why sometimes messages like those can be misunderstood, so I like to err on the side of caution over personal. Didn’t mean to say personal = bad, but that if you wish to change a broad status quo and raise awareness then communication is tricky!
The author is quite HackerNews famous, since he founded HashiCorp, exited hugely to become a billionaire, and is currently building ghostty.
nice try, Claude
lol I actually wrote all that in my own voice, that’s sad
Case in point: Amazon pressuring its workers to maximize AI use.
https://news.ycombinator.com/item?id=48148337
Like all things... this too shall pass.
My company is one. They just made "AI use" a mandatory performance goal for next year's reviews. I am thinking about retiring at this point...
Easiest goal ever !
Just use https://github.com/dtnewman/burn-baby-burn :p
Generally agree. I use AI very heavily, but rarely am I letting it actually think for me. It's a tool that reduces the time it takes for ideas in my head to manifest into reality. If you don't have those ideas, or a poor understanding of the system the AI is working on, you're going to produce slop. If you can't recognize this slop, you're more susceptible to having psychosis.
this AI transitionary phase to Quantum, light chanels and new way computation will be architected will in the future be looked at similar to a toxoplasmosis like societal wide parasite, which invaded the host in order for the host to act more favorably to it
We built too many layers of abstraction, so much that even the people in power have forgotten where the fantasies are. The objective reality is behind so many curtains that we forgot what is powering the whole theatre play to begin with. Or maybe we know but became too far detached from it to care. If you are at the same the better and the player, then what’s left?
The problem is that the only thing that has proved out so far is cyber security. Unfortunately cyber security improvements is not going to improve living standards, and it's just going to increase the cost of just doing business. There is no productivity boost, in fact it's the opposite.
What we need is automated research that leads to real results. This is possible, but it has yet to prove out. I am concerned that unless the AI companies focus entirely on this, it may be a while before we actually see true benefits from this.
What's worse, is there is an urgent and desperate need for automated research, as we have been seeing diminishing returns in human produced research for some time now: https://web.stanford.edu/~chadj/IdeaPF.pdf
I think one factor is AI is encouraging people to turn off their brains.
It sometimes feels like AI chatbot use is like the doomscrolling of work - it's always easier just to dump something into the chatbot than think about it.
The real question is: what's the fallout going to be after the dust settles? My guess is that the explosion of codebase entropy now underway from this is going to make for an interesting future - once it reaches the point where AI agents are spinning constantly despite progress grinding to a halt.
And they're be no veterans who know the codebase deeply to step in and fix things because it was all vibecoded - and then what are companies going to do?
I think that's the point where they turn back to the thinkers for help.
Deliberate thinking will be a think soon in our field.
This specific psychosis is driven by peer pressure. If you are seeing everyone around you using tools to "enhance" their work, you wouldn't want to be "left behind". So it has permeated the entire ecosystem. The lucky ones are those who are outside this (or can afford to be outside this) and can see why it isn't working. You can't be inside it and hope to have any rationality. Everyone is competing on "I can figure it out better and quicker than you can if only I can get X to work".
Very general comment/sentiment/observation here for me personally is that about a year or two ago, everyone asked me ‘so ..where is the ai’. Nowadays it seems that this sentiment is already on its way out and I see more and more ‘no ai used’ statements and non-ai workshops popping up. I am in a weird place between art, technology and ecology so I get the best and worst of these worlds. But yeah, I feel the hype cycle is coming to an end in my personal bubble. I never cared or feared ai, I mainly fear the mania around it. Luckily I am in a position where I can afford to just observe. Sure I have used AI and it helped me a lot, mainly to get projects going when I am exhausted/stressed.
I believe it. I've seen the cyber team in my org bend every rule just to get access to frontier security models.
Everyone has become like petulant children. Senior leaders want access to every shiny tool (CoWork/Codex/etc) that has some buzz around it. No one seems to care about the cost or whether we are actually realising benefits.
It's sheer madness, and you can't push back. I think AI can significantly help people be more productive, and I can see a future where they safely take on more autonomy. But we are far from that world.
I’m at a large Fortune 500 company. On a recent call, a mandate came from the CEO himself - “we have to use more agentic AI”.
And I found it really funny, because for what? Use it for what? It’s a tool. Imagine a guy coming down to a construction site where everything is progressing fine and saying “We need to use more screwdrivers”.
Companies who use AI passively and mindlessly will create immense opportunity for those who don't is a concrete definition for risk to companies, similar to the risk to individuals who go down the validation hole with AI.
This whole situation of forcing people to spend tokens is hilarious and shows who is true tech manager and who is impostor in the field. I think it’s a good indicator of competence and hopefully board members see that soon.
"In 1975, Dr. Joseph Sharp proved that correct modulation of microwave energy can result in wireless and receiverless transmission of audible speech."
I was reminded of the universe of Doctor Who. It's an incredibly complex technology, but it often behaves either unpredictably, like AI agents, or like code written by a vibe-coder without understanding the architecture or boundary conditions. And programmers are more like architects of consciousness, building machines rather than writing code.
Similarly, when the Doctor hacks a PC, he doesn't write code but rather communicates with the computer, using diplomacy to crack the agent.
It is likely that we will come to a world where software solutions are "grown" by iterations of agent work, and no one will know exactly how it works.
I think this will happen. A quick, low-quality solution is more common than a solution created by a master craftsman.
In addition to low-quality furniture, bad knives, electric kettles that burn out after a week, and poorly cut clothes that don't fit, have unpleasant fabric, and fall apart, there will also be a disposable, rotting code.
Master programmers will remain, just as master craftsmen have remained. They may even continue to earn well. However, there will be fewer of them, and the requirements for their skills, knowledge, and reputation will increase.
I use AI heavily. But my #1 productivity tool is still a custom code generator I wrote 15+ years ago. It routinely generates about 90% of the code needed to write a biz application. So using AI to write productivity tools is a great way to go IMHO.
Cloud people highly overvalue random web-services that most people (and most of the economy) simple don't rely on in any critical fashion. Yes, there are some example of people who make lots of money a day on these services, most people make pennies or nothing.
Github is not critical infrastructure. It's your personal social media addiction.
Just heard this today "things that took 4 months to build we can do in 30 minutes" like alright
Also this saying "you guys aren't going to lose your jobs, you'll just work more" it's funny