I really don't know what to feel about AI generated code. It's clear that it improves velocity dramatically in code output. It has some value to it. But what is the cost?
While I code without the help of AI, sometimes I feel like I am being left behind.
Ohh you wrote only 400 lines today? I shipped a whole app while being in the gym... hits hard.
This is what I mean, its not a low number far from it. But I know people that only value that number, so previously they churn out a big LOC, higher ups praise. Now its just big number good, bigger number even better. And LLMs do exactly that.
Now LOC comes down to how much tokens you are willing to spend for a given day.
If that makes you feel better, the guy who vibecoded an app in one sitting will probably have 0 clue on how the code works, the app itself will look like the average claude eyesore and the docs will be unreadable by humans without stimulants
Sadly, not. I trying to work with doctor's offices to improve their productivity with IT support and better structure for workers to work with patients.
You don't want to know how often I heard: "But I saw on Instagram, with AI it can be done cheaper and faster".
The people that would pay me don't care for good docs, eyesore, or if the programmer actually knows how the app works. They care for how fast are you done? And how much does it cost?
The same feeling, do AI knows the hardware? Does it have the context of the changes? Did it read Linus decisions why something should be the way he thinks is correct? I would prefer such approach where AI knows so much context as Linus and the most important persons in the kernel team
Why would you code without the help of AI, genuinely asking? You can have quite strict "guardrails" that would maintain code consistency/quality that can be close to guaranteed nowadays.
It's not only about the pace, but also about the quality, most softwares have proven that not getting AI-reviewed is risky in its own, at least in the field of security.
That's a good question. I don't have anything to critique regarding consistency or quality. You are right to say that the newest frontier models are quite good on that front. Especially when you are doing a task you have little experience in.
I like programming, it fun! Using AI feels like I am optimizing the fun out of it. I learn so much about different programming languages and topics I would simply not learn when using AI. I also like learning things!
In my case, it would be to make sure I have a good mental model of the problem at hand instead of offloading it to a LLM. I'd also rather not put myself in a position where I'm reliable on a LLM to do my job.
Oh and there are also the huge moral and ethical issues with the companies that offer LLMs.
The reality is that you will be left behind, you need to decide if thats a bad thing or not. If you want software development to be your career it probably is, if you enjoy it as a hobby than probably not. You're the 2020s version of the 1990s developer sticking with ASM.
This argument of LLMs vs manual coding being compared to higher level languages vs assembly really never holds well. To be honest people who know assembly are still definitely in demand for various open source projects as well as proprietary codebases in companies like within embedded systems, linux development, compiler engineering, etc. Whereas LLMs are a whole new class of problems and discussions with very little to almost no similarities
I code because the software i need is not there yet and these tools still fail at doing those.
Not sure why the next xy (already done) app has any economic value outside being a cheaper clone.
Still there is nothing wrong with having an AI point out problems which i assume is the case here and someone still is responsible and knowledgeable about the patched (!) code.
You are left behind.
That's like saying: "I am using punch cards for programming and I feel left behind as others are using real-time IDEs"
You don't need to vibe code (and even this is only a matter of time as llms become better and better), but not using AI at all will definitely leave you behind.
Please don’t do this if your livelihood depends on it. You’re just going to make your life more painful.
As a professional programmer, you’re selling a service. The people buying your service have fully bought into AIs and accepted their shortcomings. You can’t fight this.
You can still scratch your itch with hobby apps, or contributing to human-first projects like Zig.
The cost is similar to doing any other task in a rushed, careless manner, with little regard for the consequences.
sometimes I feel like I am being left behind.
And why does that bother you? “Forward” does not equal “good” or “desirable”. If a van is driving towards a cliff at high speed, would you not rather be left behind? “Being left behind” is a talking point to cause you FOMO, don’t fall for it.
All you need to us actually look around, has software quality dramatically increased over the last year? Have useful features been coming out at an unprecedented rate?
Because after all this talk about being left behind and I see nothing to show for it.
The code itself indeed has become cheap but the question still is out whether it solves the right problem. I've shifted my work significantly toward designing more high-level portions where multiple things would interact in somewhat unpredictable manner (read as failure modes) that's where I found the models being quite poor, they do suggest many things to solve these or that problems but in a unity these suggestions miss second order effects. I was quite surprised that models can't hold all of the connected pieces simultaneously to reason about. But once I have a complete design after a week of deliberations and various attempts, the models can produce lots of correct code to cover that. And then another pass starts because they still deviate in implementation of various pieces and once you start questioning those the models start self-correcting. Judging the output is still way faster than writing the code yourself and indeed models still show some cases that slipped once mind. It's a definitive increase in terms of quality and value of the end code no doubt. Whether it can one-shot a complex problem, not really.
Code is merely the means to an end. It does not matter who writes it as long as it does what you want. Correctly.
Stands to reason though that someone who knows programming AND has domain knowledge can get LLMs to produce much better output compared to someone who does not.
I never used to have time to make all the stuff I needed or was interested in. With LLMs, I can.
This "shipped a whole app while being in the gym" does not work for me though. It takes me a couple of days to a week to produce solid, functional software (~10KLOC). Simple tools (3-400LOC)? Yeah, those you can produce in 30-60 minutes.
The pressure to use LLMs is high in the companies that sell shovels in this gold rush. It would look bad if they did not use them. Linux developers work at those companies.
Linux kernel is a huge project - with a lot of people volunteering. You’d be burning through a crazy amount of input tokens to gain enough context to get any work done. I doubt many volunteers pay for their own top-tier subscription plans that would be able to do so cost-effectively.
I would assume that linux devs are relatively older. Furthermore they are probably very knowledgeable in that area, and one thing I've noted on here is people who are knowledgeable not liking the quality of code output from LLMs, if they have stringent quality requirements it might be easier for them to write the code themselves.
Embedded code and systems code tends to have a lot of extra constraints the model doesn't know about and aren't necessarily written down where it can see. This produces a higher rate of "superficially plausible but doesn't actually work" code.
(There are probably wider benchmarks, but this was a definite finding in our company wide evaluation of effectiveness. LLMs aren't great at RTL either)
I really don't know what to feel about AI generated code. It's clear that it improves velocity dramatically in code output. It has some value to it. But what is the cost?
While I code without the help of AI, sometimes I feel like I am being left behind.
Ohh you wrote only 400 lines today? I shipped a whole app while being in the gym... hits hard.
400 locs is a very exhausting day for me, I think I rarely did so much work a day in my entire 15 years career
This is what I mean, its not a low number far from it. But I know people that only value that number, so previously they churn out a big LOC, higher ups praise. Now its just big number good, bigger number even better. And LLMs do exactly that.
Now LOC comes down to how much tokens you are willing to spend for a given day.
Depends on the kind. 400 locs JS or Java or something is barely worth mentioning. Can do more, a lot of it is boilerplate bs.
400 locs of critical C code, sure.
If that makes you feel better, the guy who vibecoded an app in one sitting will probably have 0 clue on how the code works, the app itself will look like the average claude eyesore and the docs will be unreadable by humans without stimulants
Sadly, not. I trying to work with doctor's offices to improve their productivity with IT support and better structure for workers to work with patients.
You don't want to know how often I heard: "But I saw on Instagram, with AI it can be done cheaper and faster".
The people that would pay me don't care for good docs, eyesore, or if the programmer actually knows how the app works. They care for how fast are you done? And how much does it cost?
Leave your card to fix the mess later.
You’re a doctor. What’s your reaction when a patient rejects your diagnosis due to something they saw on Instagram?
Do they care for liability?
well even if the results are crap, it still has big economic influence:
"since I can use AI to create this app this much, I'll pay you less"
"although current AI made this crap, it'll outpace you eventually, so I'll pay you less"
The same feeling, do AI knows the hardware? Does it have the context of the changes? Did it read Linus decisions why something should be the way he thinks is correct? I would prefer such approach where AI knows so much context as Linus and the most important persons in the kernel team
Why would you code without the help of AI, genuinely asking? You can have quite strict "guardrails" that would maintain code consistency/quality that can be close to guaranteed nowadays.
It's not only about the pace, but also about the quality, most softwares have proven that not getting AI-reviewed is risky in its own, at least in the field of security.
That's a good question. I don't have anything to critique regarding consistency or quality. You are right to say that the newest frontier models are quite good on that front. Especially when you are doing a task you have little experience in.
I like programming, it fun! Using AI feels like I am optimizing the fun out of it. I learn so much about different programming languages and topics I would simply not learn when using AI. I also like learning things!
In my case, it would be to make sure I have a good mental model of the problem at hand instead of offloading it to a LLM. I'd also rather not put myself in a position where I'm reliable on a LLM to do my job.
Oh and there are also the huge moral and ethical issues with the companies that offer LLMs.
The reality is that you will be left behind, you need to decide if thats a bad thing or not. If you want software development to be your career it probably is, if you enjoy it as a hobby than probably not. You're the 2020s version of the 1990s developer sticking with ASM.
This argument of LLMs vs manual coding being compared to higher level languages vs assembly really never holds well. To be honest people who know assembly are still definitely in demand for various open source projects as well as proprietary codebases in companies like within embedded systems, linux development, compiler engineering, etc. Whereas LLMs are a whole new class of problems and discussions with very little to almost no similarities
I code because the software i need is not there yet and these tools still fail at doing those.
Not sure why the next xy (already done) app has any economic value outside being a cheaper clone.
Still there is nothing wrong with having an AI point out problems which i assume is the case here and someone still is responsible and knowledgeable about the patched (!) code.
You are left behind. That's like saying: "I am using punch cards for programming and I feel left behind as others are using real-time IDEs" You don't need to vibe code (and even this is only a matter of time as llms become better and better), but not using AI at all will definitely leave you behind.
Citation needed?
Sure you can churn out more code with a LLM, but is it good code? Is it correct? If being left behind means I don't ship slop, I'll gladly bow out.
And using AI will leave you unemployed.
Please don’t do this if your livelihood depends on it. You’re just going to make your life more painful.
As a professional programmer, you’re selling a service. The people buying your service have fully bought into AIs and accepted their shortcomings. You can’t fight this.
You can still scratch your itch with hobby apps, or contributing to human-first projects like Zig.
The cost is similar to doing any other task in a rushed, careless manner, with little regard for the consequences.
And why does that bother you? “Forward” does not equal “good” or “desirable”. If a van is driving towards a cliff at high speed, would you not rather be left behind? “Being left behind” is a talking point to cause you FOMO, don’t fall for it.
All you need to us actually look around, has software quality dramatically increased over the last year? Have useful features been coming out at an unprecedented rate?
Because after all this talk about being left behind and I see nothing to show for it.
The code itself indeed has become cheap but the question still is out whether it solves the right problem. I've shifted my work significantly toward designing more high-level portions where multiple things would interact in somewhat unpredictable manner (read as failure modes) that's where I found the models being quite poor, they do suggest many things to solve these or that problems but in a unity these suggestions miss second order effects. I was quite surprised that models can't hold all of the connected pieces simultaneously to reason about. But once I have a complete design after a week of deliberations and various attempts, the models can produce lots of correct code to cover that. And then another pass starts because they still deviate in implementation of various pieces and once you start questioning those the models start self-correcting. Judging the output is still way faster than writing the code yourself and indeed models still show some cases that slipped once mind. It's a definitive increase in terms of quality and value of the end code no doubt. Whether it can one-shot a complex problem, not really.
Code is merely the means to an end. It does not matter who writes it as long as it does what you want. Correctly.
Stands to reason though that someone who knows programming AND has domain knowledge can get LLMs to produce much better output compared to someone who does not.
I never used to have time to make all the stuff I needed or was interested in. With LLMs, I can.
This "shipped a whole app while being in the gym" does not work for me though. It takes me a couple of days to a week to produce solid, functional software (~10KLOC). Simple tools (3-400LOC)? Yeah, those you can produce in 30-60 minutes.
The pressure to use LLMs is high in the companies that sell shovels in this gold rush. It would look bad if they did not use them. Linux developers work at those companies.
Seems low. 80% people are manually typing in things? In my circle it's close to 0% code written manually.
Do linux devs live in a completely different microcosm?
How big is your circle? What industries does it span? Maybe the people writing 0% code manually are outliers and your circle is relatively small?
Small changes can have a huge impact.. far beyond your context window ;-)
My hypothesis:
Linux kernel is a huge project - with a lot of people volunteering. You’d be burning through a crazy amount of input tokens to gain enough context to get any work done. I doubt many volunteers pay for their own top-tier subscription plans that would be able to do so cost-effectively.
I would assume that linux devs are relatively older. Furthermore they are probably very knowledgeable in that area, and one thing I've noted on here is people who are knowledgeable not liking the quality of code output from LLMs, if they have stringent quality requirements it might be easier for them to write the code themselves.
Embedded code and systems code tends to have a lot of extra constraints the model doesn't know about and aren't necessarily written down where it can see. This produces a higher rate of "superficially plausible but doesn't actually work" code.
(There are probably wider benchmarks, but this was a definite finding in our company wide evaluation of effectiveness. LLMs aren't great at RTL either)
I'm curious ... how would they even know?
Huh.
Wait until it blows.
And I say this as someone who uses AI (for fun) and to check things I may havbe overlooked.
There's a place and time for everything. Using AI where you shouldn't is gonna backfire, big time.
Also, just please show me ONE single app/product that has been generated 100% by AI--that works and competes with human-written ones?
I'll be waiting, right here.
So the Linux kernel has now just become by definition "slop"? I mean, there is a definition here which says it is now "tainted". [0]
Does that mean we should stop using it?
[0] https://codeberg.org/ethical-foss/open-slopware