- 36comments
- 27comments
- 16comments
- 185comments
- 97comments
- 341comments
- 37comments
- 60comments
- 22comments
- 4comments
- 120comments
- 5comments
- 6comments
- 19comments
- 108comments
- 859comments
- 1comments
- 295comments
- 18comments
- 8comments
- 12comments
- 96comments
- 77comments
- 269comments
- 19comments
- 284comments
- 131comments
- 395comments
- 35comments
- 55comments
You are holding it wrong!
I think there's a lot of setup and context required for an AI agent to consistently write good code. Once the agent has these guard rails in place I usually get great quality- far better than what I would write in most cases.
I think where things get dicey is being able to write in any language. I write and review code in many languages and frameworks I'm not fluent in, so it's hard for me to distinguish between working code and great code. I can spot when the fundamental logic is wrong, but when it comes to "best fit" choices I'm clueless.
The issue is that in order to have the agent write good code, you need to implement standard SWE best practices. But that also means a lot of manual intervention in terms of writing specs, checking acceptance criteria, and reviewing code. So you end up spending a lot of time on managing your agent, which means you won't get a 1000% productivity gain, you get maybe 50 or 100, possible less in some areas and with some issues.
50 or 100 seems unlikely. Even with all these improvements, custom setups and guardrails it just isn't that much faster for me.
I'm curious as to what guardrails you've tried.
This is something I have been trying to get right as well. I've attempted to use lots of linting and things like strong typing, duplicate checks, cyclomatic complexity, and robust tests. However, I still happen to find issues, which requires me to look at the code (at least at a high level)
For example, I can say "Don't repeat yourself, and don't re-write helper functions" and I will even have a duplicate linter check, but inevitably the LLM will always want to re-write a similar yet slightly different helper function. Like it will always want to re-write something small like a trim() or a toString() function in every file.
[delayed]
I find that if I leave an instruction in AGENTS.md to "do not do X", there's a good chance the agent will forget it.
But if I add a separate post-implementation review/fix pass by the agent, it'll usually find and fix the issues.
So I've started doing it for everything from naming conventions to duplicate code to other problems. It does cost more tokens, but now I get less frustrated at having to fix basic issues in the PRs.
If AI is not lowering your code quality, you weren't very good to begin with. The point of AI is to increase your productivity tenfold while maintaining acceptable (but not great) code quality.
Ah, the "skill issue" argument again. Same crap aswhen everyonewas worshiping Musk 5-6 years ago, this time it's dario and altman with a claude/chatgpt mask. Crash can't come soon enough.
I think the point is just it doesn’t have to get worse, so there are things you can do to prevent / change it if it is deteriorating.
Yes, but it doesn't matter if nobody actually does that. Either because
1. they don't care
2. the rest of the team doesn't care
3. the powers that be actively discourage it because velocity.
That's a fair point. I guess step 0 is that you have to care about code/product quality and prioritize it.
And why wouldn’t writing software be a skill issue? Yes, it’s an annoying meme, but we should expect that there are better and worse ways to write software. It would be weird if everyone got the same results regardless of experience.
I’m doubtful that the author’s recommendation always work, but I do some similar things and they do seem to help.
Anyone who thinks they can produce high quality code from an LLM is mistaken about how to judge code. Trust me, I've seen enough PRs to last a life time. A lot of professionals wouldn't know good code if it slapped them in the face.
In a way it reminds me of the good old "if agile doesn't work for you, you're not doing agile right".
It is indeed skill issue.
You don't think crash will happen because XYZ. You _wish_ for the crash because you are hateful of progress that you are not part of.
AI writes unmaintainable code - you can see that many projects don't accept it.
To be fair, so do humans.
Yeah but in my experience AI boosts productivity of those humans 10x and only boosts productivity of programmers who do write maintainable code 50-100%.
There also exist other good reasons why projects don't want AI-generated code, in particular
- because of unclarity of copyright status and consequences of AI-generated code
- because the project leader simply made the observation than many programmers who hand in AI-generated code care more about "getting things done" and "pushing through their changes" (possibly to boost their CV) instead of deeply caring about code quality
If AI coding isn't lowering your code quality, you're not using it enough
I'd say step 0 is know your audience.
I'm happily vibing my own toy projects, but would prefer if the tech in hospitals is not vibe coded.
And I don't think it's plausible that the gap between those two is "well you just need to use it right".
If AI coding isn’t lowering your code quality, you have a low starting point.
Of course, it's your fault, not LLMs not being able to write good code and destroying whole codebases in a matter of weeks.
You can have all the measures in place that are described in that post, and your code can still be bad. High unit test coverage tells you exactly zero about the solution itself.
And technical quality gates do not help if the human side lacks defense against slop code. If you don't have the right managers in place, the 2 years of experience vibecoder who ships a feature in 4 hours will always win against the 20+ year senior who actually looks at the code he is about to ship.
A sibling comment talks about needing a lot of setup and context for agents to produce good code. That’s both true and bizarre.
If the compiler that I write produces lousy code, I get bugs that I fix until it doesn’t.
And that is the most annoying thing about this revolution. It’s obviously powerful and transformative and I use in my job all the time.
But many, perhaps even most, purveyors seem intent on blaming their users when they have issues, rather than fixing their own bugs.
General model improvement is going a long way here, but basic things like “ensure you use good style and programming practices” really shouldn’t be a thing users need to put in any .md file.
A programming language spec is expected to be unambiguous. A compiler is expected to be deterministic. There are multiple ways to different outputs when compiling (optimizations, etc) but those are also meant to be well defined and deterministic themselves.
AIs are stochastic/probabilistic machines. Their big potential is in how they take malformed, incomplete, ambiguous inputs and come up with valuable and usable solutions.
Eh. I wouldn't focus on unit test coverage.
I think it's true that good, well tested code will have higher code coverage than crappy code.
But, above a certain point (which will vary from codebase to codebase), unit tests aren't meaningfully increasing confidence that the code is working.
I'd recommend focusing instead on the code being written in a pure 'functional core, imperative' shell to the extent that's possible. For that pure/functional part, 100% code coverage is attainable (& so not worth remarking on). For the impure parts, unit tests are probably using "mocks" just to get the code to compile anyway.