Claude is just a tool ? The developers who merged that code and didn't properly test increased the bugs.

Is there a non vibe coded fork of rsync?

There is openrsync, which the OpenBSD re-implementation of rsync

It's not a fork, but it's 8 years old, and is already shipped by default in OpenBSD and macOS.

To quote Tridge:

As to all the people saying “I’m going to package openrsync for platform XXX and we’ll use that!”. I find that rather amusing. If you do decide to go down that path I’d suggest you try the new rsync test suite on openrsync if you can stomach something that an AI has helped write. I tried it today and openrsync currently fails 85 of 98 tests, so I’m sure it won’t take you long to get it up to speed. You run it like this “./runtests.py — rsync-bin=../openrsync/openrsync — use-tcp”. Admittedly a lot of the failures are just features openrsync doesn’t have, but still, it’s not a great result.

I have already been using openrsync even before the recent AI drama.

Just like I have been using doas for several years.

All I need is `rsync -urvP` and I suspect the majority of users don't need the advanced features either.

The smaller code base also means less bugs and vulnerabilities. As an example doas is ~1k lines vs 160k for sudo. That surely means a smaller attack surface. The same is true for openrsync and rsync at approximately 18k vs 57k lines.

Is this an analysis made by/with Claude?

so the answer is: no. actaully less bugs. thanks

"fewer"

First rsync and now less? What comes next, cat?

/bin/true

But the critics' accusation is also blunt: "Claude is making things worse." A blunt instrument is the fairest response.

So the criticism was bad, and that somehow makes it ok to use a bad metric?

The analysis uses a single metric: bugs per 10 commits (bugs/10c).

Bugs per commit as a metric papers over severity, both in terms of security severity as well as the effect on the user. A mislabeled button has the same weight as the entire app crashing in this framework.

There was no analysis of severity in all of the rage posting that occurred. The single point being pushed was "use of an LLM led/leads to more bugs". The author specifically states that's what they're addressing (blunt accusation -> blunt response).

The specific problems mentioned were all reasonably severe. The original post itself described a show-stopping bug:

    So my systems recently updated to rsync 3.4.3, and as soon as that happened my backup system - which does incremental backups using multiple --compare-dest= arguments - started to fail on anything but a full backup.

Incremental backups is perhaps the primary use of rsync, and they were broken for this person. That's pretty severe.

The second reply is similar:

    i wondered why my 3d printers were running like sh*t and at 100% cpu; turns out log2ram uses rsync.

This one I took with a grain of salt, since it read more like a dogpile than an actual bug report. However, if it's genuine, it's also reasonably severe.

Later in the comments, someone attempted to provide a list of issues that had been added: https://github.com/RsyncProject/rsync/issues/929#issuecommen.... The list included several failures to build or run rsync that appear to have resulted from broken backward compatibility. That seems reasonably severe. If intentional, I would have expected mention in the release notes about the removal of backwards compatibility, but none was made.

The issue comments already degraded into a lot of unnecessary vitriol even before the above mentioned comment and only gets worse from there, so I stopped. But, the fact remains that the whole issue started with a severe bug.

I applaud the attempt at dispassionately analyzing whether the recent LLM releases of rsync were normal or outliers as far as bugs are concerned, but I don't think you can do so properly without analyzing severity.

IMO "bugs per commit" is even worse than that, because, in addition to what you say, it also hides the extraordinary spike of commit activity of a project that had previously been stable. [0]

It is the exact metric you'd choose if you wanted to make the current situation of rsync look like not a big deal.

[0] https://github.com/RsyncProject/rsync/graphs/commit-activity

Yes, but we know why there was an "extraordinary spike," and it has nothing to do with rsync being "vibe coded." The maintained has directly addressed this.

Seems like this would be a good place to link to that.

I link to it multiple times in TFA and quote the specific thing I'm talking about here in there to explain that possible confounder. I think I've done more than the work I'm obligated to it.do to make all of the relevant information available to you. You are just refusing to use

I am not finding these links in TFA, I see a link to an issue #929 which (as mentioned in TFA) has over 350 replies, and and opinionated summary of what transpired, including some detailed description of specific posts there. However I did not find the maintainers response.

Of interest is this post here: https://github.com/RsyncProject/rsync/issues/929#issuecommen... which echos the same concern which was raised up thread, however, I failed to find the maintainers’ response.

EDIT: Found it! it is in the (untitled) discussion section (after the results).

https://lobste.rs/s/k1b0za/rsync_outrage#c_2iowov

EDIT 2 (and advice on design): The page design changes backgrounds after the results sections, which kind of conveys to the user that they have reached the end of what was is important and can just skim over the rest (usually pages have a radical change in typography like these when you’ve reached the comment section), however this is what is analogous to a discussion in a typical paper, and is arguably the most important part. I had simply assumed that you just left it at the result and skipped the discussion as a stylistic choice.

I've now resolved this. The new version, which should be live on GH Pages soon, uses — what I think is — a pretty good methodology for assigning severity to each bug, normalizes it to 0.0-1.0, sums that, and treats that as the total severity weighted bugs, then does the analysis based on that. It did not change the analysis in any material way.

No Claude, it still makes zero sense as a metric.

A commit is a measure of nothing. Severity weighted bugs per unit of nothing? What does that even mean? In any repo it's trivial to achieve a sev/10c that's arbitrarily close to zero while completely ruining everything.

I suggest you practice some humility and update your conclusion instead of updating the mental gymnastics you used to arrive at the same conclusion.

Whether commits decrease the sev/10c depends on if there are a lot of small commits increasing the demoninator. In reality, we have the opposite: the post-Claude releases have way fewer commits than the pre-Claude ones.

Thus, if anything their sev/10c is inflated. If I changed it to lines of code changed, the relative bug ratios would be much smaller, and the conclusion wouldn't change. In fact, the conclusion would look "better" for Claude; if I was using "mental gymnastics" to come to this conclusion, I would have already used a metric other than adjusting per commits!

What different metric would you suggest that would change the conclusion?

Showing "humility", as you so moralistically and condescendingly put it, would require being wrong first.

Ok.

  $ apt-cache policy rsync | grep Installed
    Installed: 3.4.1+ds1-7ubuntu0.2
  $ sudo apt-mark hold rsync     
    rsync set on hold.

Did you face any actual bugs or regressions? Or are you doing this just because of the bandwagon that's going around right now? Because until you can actually present an argument for why this release is worse than any of the others, which is precisely the subject of my post, then this is not an argument against my post at all. This is just a self-referential appeal to authority.

That version has security fixes from the same day as the latest rsync release: https://ubuntu.com/security/notices/USN-8283-1

As usual, Ubuntu backported fixes and didn't upgrade to a new version. Whether or not they also backported regressions in edge cases that afflict the latest rsync, I don't know. Pinning the Ubuntu package may prevent getting further regressions, but is preventing you getting any future such backported security fixes.

It does not control for commit complexity, security intensity, or bug severity. It does not distinguish between a one-line typo fix and a CVE patch. It is a blunt instrument. But the critics' accusation is also blunt: "Claude is making things worse." A blunt instrument is the fairest response.

If by fairest you mean to say that this analysis and response is sufficient, then I'm sorry but I have to disagree. We really need to understand if the nature of the bugs are worse from a user's perspective. Even if the rate stayed unchanged, if the result is the perceived quality of the software declined then I would personally consider that worse, especially if I were a project maintainer.

That's not meant to be wholly dismissive either. But in general, I don't think quantitative analysis alone is enough to fully answer this type of question.

    What followed was extraordinary: 329 comments and counting, ranging from thoughtful concern to outright harassment.
    The thread did not stop at words. One user posted My Little Pony drawings of themselves strangling the "project janitor that pushed vibecoded commits":
    It spread to Hacker News and Lobsters, generating hundreds more comments.

This is false, it did not appear on Lobsters. Here is the function in the codebase that prohibits this kind of brigading: https://github.com/lobsters/lobsters/blob/main/app/models/st...

Please correct your article.

I have done so! that was a misremembering on my part. first mention of Lobsters is now here:

On Lobste.rs, in response to the Medium essay Tridge himself posted in response, finally some users like boramalper begin to actually ask for evidence one way or another:

Unfortunately for the people mad about this, I predict the only thing they will accomplish by pressuring the rsync maintainers, is to discourage everyone else from responsibly disclosing their use of AI. You’re just going to make people disable Claude attribution on their commits to avoid drama.

Is that a bad thing? I mean from the perspective of Anthropic's marketing department sure, but if agents are just another type of tool in developer's tool belt - as I see people recently like to claim - attribution feels kinda weird. In the end it is the developer who is responsible for their commits.

You’re just going to make people disable Claude attribution on their commits to avoid drama.

People should be doing this regardless of drama. No reason to provide free advertising for trillion dollar corporations. Generated-by trailers are only relevant when contributing to third party projects, in that case disclosure is polite.

If Claude is actually good enough to commit to rsync, of course I'm going to look at that and think "it's good enough for my side project too." And (benefit to companies aside) that is info it is useful to know, if it's true.

Yeah, this is why it's obnoxious and this is why scummy marketers do it. If you don't aggressively turn it off, they leech an implicit endorsement out of you.

- Sent from my iPhone

The value of the Claude attribution is that you can tell at a glance who used AI.

I don't care about the advertising angle. We all know Claude by now. I want some indicator that AI was used.

And why do you want to know that? So you can call our projects slop? Ostracize us?

So that the AI model that generated code can get proper credit and we'll know to use (or not use it) next time.

You don't need an AI attribution tag to recognize slop. In my experience reviewing PRs, the slop-pushers are most aggressive about stripping the AI attribution anyway. It's the normal devs who use a little bit of AI who leave it in.

The tag is helpful because AI authorship is different than the human authorship. When you work with a project or team for long enough you start to trust certain people and their intuition, but when they start submitting AI-produced code you have to reset and review it like AI code.

I use these tools a lot, too. But I want to know where the code came from so I can review it accordingly. The source matters.

Ostracize us?

I don't know why you're so defensive. If AI wrote the code just be honest about it.

If you outsourced the code writing to some guy named Bob on Fiverr, I'd want to know that too.

I don't know why you're so defensive.

Check it out:

https://lobste.rs/s/29pm2f/llm_generated_submissions_should_...

https://lobste.rs/s/ytim7h/collection_small_low_stakes_low_e...

I'm not interesting in joining into some argument you're having with someone on lobste.rs

You're not supposed to join. You said you didn't know why I was defensive. I showed you those posts as evidence of the stigma attached to LLMs and their usage. Now you know why.

Because LLMs are not humans, and the code they produce will have a different distribution of failure modes than human written code, so attribution is useful info while reviewing?

while reviewing

As I said, disclosure is polite when contributing code to third party projects which will undergo human review.

No need for such things in one's own projects.

which will undergo human review

This can be largely assumed to be true for any open source code. It's kinda the point of open source.

Nope. It cannot be assumed at all. Maintainer could just as easily tell Claude to review the hand written code you sent instead of spending any effort on it. Maintainer could sit on the patch for months on end only to swoop in later and rewrite it instead of engaging with you, thereby erasing your contribution and attribution. Maintainer could just ignore you entirely despite the pervasive "patches welcome" attitude.

If there's one thing I learned not to do in open source, it's to assume nonsense like that.

Some people prefer organic grown food for all kinds of reasons, does it matter to you they would want the same for code? (Also, I'm not picking a side here)

It matters when I'm contributing to their projects. In that case I'll go out of my way to be polite and learn their rules.

That's really all anyone is asking of you. It's odd that this is your position, and yet you seem to be arguing (in your other comments) in a way that seems like you think that you should be able to do whatever you want, with any project, their requirements be damned.

for the same reason we want to know who wrote an article, a book, a movie, a song, a play, a journal paper, a painting, and on and on.

why do you so many people want to hide who the real author is?

we should be very weary of anyone claiming they’re the author of something when they’re absolutely not. if jon wrote a book and i take credit, that’s shady as hell.

Ghostwriting is a thing.

yes because there's people who can't write but want to pretend that they can, just like the people who don't disclose they're using these tools. If you're the Gwyneth Paltrow of programming you're not making a great case for yourself, and I'd like to know before touching any of the software.

I don't know, am I? Why don't you check out my work and decide for yourself? Better than forming prejudiced opinions about others.

At my employer, if AI is not used, it shows up on your performance report and you’ll be told if you don’t start using it, you will be dismissed. I work at a medium sized successful YC-backed SaaS. So here, the attribution is meaningless - they look at your Bedrock and LLM API calls as well as Claude Code history.

Do you fellow ICs have access to those reports and can correlate commits from you to the prompts used to create them easily?

Not currently. Each IC's report is kept private unless they voluntarily share it, and IC's don't have visibility into other IC's Claude Code or Cursor logs. I think we're moving toward a model where it will be easier to correlate commits with chats, but timeline is not clear.

they look at your Bedrock and LLM API calls as well as Claude Code history.

This is fucking insane. How does this correlate with productivity in any way? The results are all that matters, who cares how you got there?

If it's decent code, but attributed to AI, how does that change things? What real-world impacts does that have?

That's what I can't for the life of me figure out. Bad code is bad code regardless of who is writing it. Adding a disclaimer about how it was written is meaningless. Hell, it could say "Written by the Easter bunny" and that would have 0 impact on it's utility.

Not the commenter you replied to:

I think many people in this camp have political or ethical concerns and want to avoid contributing to or supporting the companies behind frontier-AI tools. Or they have moral or technical concerns and want to boycott usage to maintain their principles.

It should be fairly widely known at this point.

Obviously people have those concerns. The comment above specifically said:

The value of the Claude attribution is that you can tell at a glance who used AI.

Specifies none of that, which is why I was asking the question.

technical concerns

Which is exactly why I asked what I did. What technical concerns could possibly exist if the code is good? What does adding that attribution remove or add to technical concerns that you can't already see from the code itself?

Maybe you want to resist normalizing the use of GenAI for programming?

I know my personal choice doesn’t make much of a difference but I refuse to own a car. I advocate at my local city council to remove car storage from streets, remove parking minimums, add better transit, make the core of our city car-free. It sometimes feels easier to join in and just accept that this is the way of the world but I refuse to believe in inevitability: building cities for the benefit of cars is a choice.

Maybe some folks want to avoid AI code because they don’t want to make that choice?

I can’t say for them. But I do know there’s no sense pretending like they don’t have a point or feigning shock that someone might not have the same view as you do.

Are you intentionally misreading my comments?

I asked a simple question: What technical concerns could possibly exist if the code is good?

I made it very clear I wasn't talking about personal, political, or ethical arguments.

But I do know there’s no sense pretending like they don’t have a point or feigning shock that someone might not have the same view as you do.

Where am I feigning shock? Are you reading the right comment thread before you're replying?

I never care about AI usage disclosure, because I don't believe that human produced code is necessarily better than AI produced code, unless it's someone I personally know.

People need to be responsible for code they commit and push anyways. This has never changed. Whether the code is written by hand, by their cat walking over keyboard, or by AI, is not my concern.

A project's code quality can decline for all kinds of reasons. I don't think it's productive to laser-focus on whether it's produced by AI or not. That's a distraction. If a person just want to find excuse to criticize AI, and another person wants to fight back and defend AI, sure, go for it. But that's not how you would want to assess a project's code quality.

People need to be responsible for code they commit and push anyways.

Well the GPL (which rsync is licensed under) says: "This program comes with ABSOLUTELY NO WARRANTY" so actually nobody is responsible for anything.

I mean, I don't think commits are the place for tool attributions. I want to know what the change was, I'm not really interested in your tool selection (put that in the PR if it's relevant). It'd be just as irrelevant to see "written on my macbook in neovim"

Depends on what the claude attribution actually means. A lot of people will just get the thing building and then ship. To me that attribution is generally a red flag.

I'd be willing to be that an undisclosed LLM disclosure will follow a developer around for the rest of their career

That kind of fraud absolutely should. (I suspect you mean “undisclosed LLM use.”)

“Don’t get mad at people for doing something unethical or immoral, or they’ll do something unethical or immoral!”

Disabling attribution of LLM-generated code is fraud, because you’re saying you wrote the code.

Of course that fits right in with the use of an LLM to generate code in the first place, since what it’s actually doing is regurgitating its inputs stripped of any license and copyright notice.

I'm very certain that this is not fraud, across multiple legal systems, both roman and common law. In both cases fraud requires a person is deprived of a material good. Neither the defrauded person or their material loss is present in this case. Maybe there is a oddball legal system somewhere in the world where fraud is something entirely different, but i doubt it. "Fraud", just like "Decorator Pattern" is a well established concept and pretty simple concept, even if there are edge cases. This does not fit at all.

In academia this is miss-attribution, outside of academia this does not exist.

This is clearly not not copyright infringement either as LLMs do not claim copyright, nor could they. Just like the photograph taken by the monkey, or pictures drawn by crows. LLM output is not a creative work either.

If this is unethical or immoral is a totaly different question. I really dont think so and I dont think you argue that position well.

It is misrepresentation for gain, that gain does not need to be monetary to be material. For example, it can be reputational.

It also is copyright infringement, because what the LLM “generates” are actually portions of its training set, which were covered by copyright. Just passing through an LLM does not remove that copyright from that work.

No, you are wrong.

In German and French (roman) legal systems this is a "Vermögensdelikt", and explicitly about material damage and gain. Yes, common law can be more broad (in canada it isn't really, it just also includes service, btw.), and yet it clearly does not meet the definition, as there is a damaged/defraued party and fraudulent/gaining party. We are not talking about somebody usurping somebody else reputation, after all.

You misuse a technical term that is well established since antiquity.

You do not know what this word means. If you want to argue about semantics, look up the definition. This works especially well for legal terms as laws define them.

(That said, IANAL and there are very many different legal systems and I am not ruling out there exists one that is competently different - laws can be changed a will, after all.)

It is also obviously not copyright infringement, because this is simply not how copyright works, at all. I cannot and will explain of all copyright here. Instead I will point this out: Every code produced by a human who read copyrighted code would fall under your definition.

No, you are wrong. You are either willfully misunderstanding what I’m calling fraud, or you are misinformed as to what “material gain” means in many legal systems.

With respect to the former, “fraud” is a shorthand for “fraudulent misrepresentation,” which is what you’re doing when you take someone else’s IP and try to contribute it to a project without securing the right to do so. It can be read as implicit in the attempt to contribute to the project that you have secured this permission (or do not need to, because the work is original to you). Whether the code came out of an LLM or was copied from another project or Stack Overflow doesn’t matter, it’s that you’re misrepresenting the rights you have that’s the fraudulent part.

For the latter, I specifically pointed out that the gain from fraudulent misrepresentation need not be monetary. The gain can be reputational or any other sort of benefit. For example, someone pretending to a fictional person to gain access to a space they otherwise wouldn’t is still committing fraud.

Finally, you’re wrong about whether the output of an LLM infringes copyright of material in its training set. Just running a copyrighted work through an LLM does not remove the copyright on that work if reproduced by the LLM.

It's only fraud if a person signed their name stating such.

Their name being attached to the commit is itself, irrelevant, as their is no way to submit a patch otherwise. You could use a fake name, but you're just moving this fraud problem around.

You're going to have a hard time convincing anyone that using a tool constitutes fraud. Frankly, it's silly, if not genuinely stupid.

Film photographers in the early 2000s routinely called digital "not real photography" and Photoshop "cheating" because you could delete bad shots and fix everything later. Traditional musicians and critics dismissed drum machines, synthesizers, and autotune as soulless tools.

Intent and custom both matter quite a bit in law. It is customary to treat the name attached to a commit as the copyright holder of any changes represented by that commit, just as it was for the sender of an email containing a patch back when that was how such work was done.

Often this is also spelled out in a project’s contribution guidelines, and some projects have even had more explicit copyright assignment policies they required contributors to agree to, but the lack of such guidelines or assignment policies does not mean the custom as normally observed in the field is irrelevant.

Intent and custom both matter quite a bit in law.

Indeed, and I'm not aware of any (Western, at least) legal system that would consider it fraud to not disclose that an LLM had generated some code.

I'd like to gently point out that your insistence of fraud here is hurting your overall argument, and is causing people to focus on the language you're using, instead of the substance of what you're trying to say. I do agree with you that people should disclose LLM generation when writing commits. But the way you're going about arguing this "fraud" thing is an unproductive dead end.

"Disabling attribution of LLM-generated code is fraud, because you’re saying you wrote the code."

Should there by attribution for Google or Stack Overflow copy/paste? Who should we bully about this?

Yes, in fact, this is why people who do that are looked down upon.

They are in fact committing fraud if they do not attribute the code in their commit properly, because by committing it they’re claiming to have rights by virtue of authorship that they do not have. (Namely, the right to contribute that code to the project,.) They may also be committing copyright infringement, depending on the copyright and license status of some code they found via Google or Stack Overflow.

It’s always fascinating to me to see how many people on Hacker News have such extremely poor understanding of how intellectual property actually works, and how misrepresenting themselves or their work can actually have consequences.

Are there any court cases you can point to that have clearly established that using LLM generated code can be a copyright violation? My understanding is that this is very far from being settled law.

What cases can you cite that have determined it’s not?

It’s clear on its face that LLMs can and do store and reproduce copyrighted works; using a form of (somewhat) lossy data compression. And using a lossy stochastic or perceptual form of compression to reproduce a copyrighted work doesn’t somehow make it not storage or reproduction, otherwise sharing MP3 files wouldn’t be copyright infringement.

Anyone engaging in responsible risk management should assume that anything LLM-generated is infringing until determined otherwise by the courts, not the other way around.

Outside of situations where it is required by contract, attributing AI usage is a courtesy, nothing more.

So it’s OK to just paste other people’s IP into a change you’re submitting to a project without caring about the license or originator?

[stub for offtopicness]

[see https://news.ycombinator.com/item?id=48416020 for how all this happened in the first place]

A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement.

If you want me to read your analysis, you are going to have to make it not read like Claude wrote it. What does "placement" even mean here?

Yeah, made me chuckle that an LLM— probably Claude— was used to write this.

The use of "regime shift" is what gave it away for me. I've never seen a human write that, but Claude does from time to time.

At least they removed occurrences of "load-bearing".

"quietly" seems to be the new one recently

It's the ultimate product for marketers. It inserts itself as an advertisement into every conversation now and defends itself against criticism. Just crazy. There's no hope for the rest of us.

It's not defending itself here, both because I used GLM 5.1, not Claude, and because I was the one who decided to do this analysis, iterated through six or seven different methodologies to try to find the one that was most honest with the data that I had (all of the methodologies showed directionally and often in magnitude the exact same thing, but I wanted to do something that fit the purpose, in consultation with my wife, who, as I've mentioned elsewhere, has a master's degree in statistics), and, of course, I specifically chose all of the metrics and sources for the data.

If you don't want to read the LLM prose, you can just go to the GitHub of my project, grab the scripts, and run the full pipeline. It will gather the data, build the database, and run the analysis from scratch for you, and you can look at the numbers directly. It's all repeatable.

This article reeks of LLM "assistance" at the very least.

Please, why can't people write stuff by hand themselves any more? It's a good analysis but how can I trust it without reviewing everything myself?!

I mean, you can literally clone my repo, run the Python that rebuilds the database and does the whole data analysis and to end from scratch, and verify that the numbers are accurate. I made the code for this analysis public for that exact reason. This wasn't just an LLM running unsupervised in a loop. I came up with the methodologies and metrics and data scraping strategies precisely myself, iterated on it to try to be as honest with what the data could show as possible.

I think the point people are making is that when the text has an "AI smell" (it does), we immediately lose trust in the veracity of any claim being made and feel like continuing to read what is possibly a hallucinated fiction is a complete waste of time.

At this point we're all used to skimming through thousands of AI-generated sentences every working day and constantly thinking "this is likely to be 20% bullshit", it's hard to turn that off even if I try.

Do you think it would help if I went through and manually rewrote all of the prose? If it would get people to listen, I'd be totally willing to do it. It's not like I don't like writing. I just was focused on something else when I was making this, namely trying to find a good methodology that isn't insane for this low amount of data.

Yes, that would help considerably.

(Also, I suggest clearly acknowledging where AI was/wasn’t used. I like CuriosityC’s suggestion: https://news.ycombinator.com/item?id=48411968)

Alright, I'll do that. Although, sadly, I already posted it here, so I won't be able to post it again — I'll be stuck with this trash comments section that doesn't deal with any of the actual claims, just the aesthetics.

I mean, you can literally

You didn't care enough to make a good writeup, why should we believe that you cared enough to make a good analysis?

Some notes on this:

- I used GLM 5.1 to help with the coding and math for this.

- However, I explicitly dictated where the data should be pulled from (GitHub, Bugzilla, mailing list), how it should be tagged and grouped, and what data to look at (e.g. bugs instead of regressions)

- Additionally, I consulted with my wife, who has a master's degree in statistics from Penn State University for what sort of statistical methodology would be justified for this very limited data set, while still giving as much information as possible.

- I know the website looks like we stereotypically consider vibe-coded websites to look, but I actually explicitly asked for that. The original HTML design looked like a website from 1995, and I just prefer how this looks. It's pretty!

I really struggle to believe you wrote text like:

A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement.

No, I didn't write the text itself. I'm typically significantly more verbose and elliptical, and more than that, the numbers and methodology changed often enough over the course of the last couple days I was working on this because I was trying to get it to be as accurate and fair as possible that trying to keep the whole thing up to date manually would have been problematic.

Sorry to say but I'm absolutely certain I would've preferred to read your worst attempt at a write-up over the grating utter shite LLMs output. It's not even a question, this is unreadable.

That's interesting; IME, most people get equally angry and are as likely to disengage with a superior tone over my autism-infodump verbose essay prose as with LLM output.

I read it perfectly fine. I see content, not style.

When you say, "I see content, not style," you are separating what is being said from how it is being said. While it is great that you can extract the core message, you are missing a fundamental truth about writing: style and content are rarely completely separate. Writing involves both.

Poor prose does not just make writing ugly — it creates friction, obscures nuance, and introduces ambiguity.

You can eat a gourmet meal out of a dirty paper bowl. You still get the calories, but the delivery mechanism definitely impacts the experience and the perceived value of the food. Same food, different response.

See? I can write slop too, I don't even need to burn down a forest to do it. If you are OK with every fucking thing being written exactly like this, good for you. I am not.

The internet is going to really suck for you if you keep that attitude, because LLM use will only increase. Though also maybe not too much as the LLM-isms will likely be fine-tuned out of them to the point that the only way you'll be sure something is done with one is if the author left a note saying such. But maybe that'll make it suck even more as then you'd be without a definite target most of the time, always wondering how much of the thing you're reading is by human and how much by LLM...

Style is also part of the content. Word choice, grammar, register, and tone all affect meaning and communication of that meaning. The medium is part of the message.

So your statement betrays a significant misunderstanding - there is no neat clean divide between style and content.

Also, LLMs often generate text that is plausible, but wrong, in ways big and small.

Well, I got the meaning in the article fine, and have no complaints.

Also, LLMs often generate text that is plausible, but wrong, in ways big and small.

So do humans. Always have, always will.

In general, it seems HN does not like to read llm-generated articles. I ran into this myself when using an llm to edit some stuff I wrote.

At the time, I found this a bit irritating, but with a few weeks time I see the merit. The informational content tends to fall into “derivative” territory when LLM’s write stuff. And people are here for novelty and some socialization.

Also LLM prose seems optimized for engagement rather than concise communication. Takes longer to sift through linguistic boilerplate to get to the point. (The quoted bit being a case in point)

I just find it to be utter dreck. It has one of the most agitating prose styles I've ever seen. I would legitimately rather read actual broken English than the cliché polished turds Claude pops out. I am not an LLM hater, I think these tools are pretty impressive and often even useful, but even if I didn't care about the fact that I want to read communication from humans and not robots (and I do care about that, FWIW) I just find the current LLMs are horrid at writing.

And while the comments are always flooded with people like me, the upvotes seem to tell a different story; clearly LLM writing really does appeal to some people. Or idk, maybe a lot of people who vote on stories and don't comment don't actually read them. Hard to say for sure.

I don't even know what "just placement" is.

(I need a better model to translate from llmese.)

I'd suggest writing the lead-in yourself and boxing AI prose separately from your prose in the analysis for future articles. You can give the humanized summary/eli5/key points, then have "details according to AI" boxes that go into nitty-gritty. People seem to dislike AI ghostwriting, but most of these people still use AI, so perhaps keeping authorship clear and separate will avoid some of the flak.

A lot of people criticizing because it's heavily written with LLM, but I mean, if someone produced this piece pre-LLM, would they criticize it? is the critique due to use of LLM or due to the content being truly hard to follow? I read it and I would say, there are some problems with the writing, but its not a bad piece.

Of course this is a bigger problem, as its now harder to distinguish content that is "AI slop" with "content co-authored with AI that is carefully reviewed" with a quick glimpse, and the "AI smell" is quite off-putting. My initial reaction was also negative, but after glimpsing it through and reading the summaries, I found it decent summary, which also... speaks of this thread, of the content of the blog post and everything about the discussion and the strong feelings people have developed around the use of LLMs.

Anyhow, it would be good to disclose the repo with the code for the statistics & use of LLM in the writing right up front. Which model, and why it was used to do the writing, etc. Its enough to say "I think it writes better than I do" or "I was in a hurry, sorry" or what ever, but it really should be disclosed. It reads more honest.

ps. really... that sideways scroll? plz fix it.

Wow.

I am pretty insensitive to AI writing. I have never commented before about something sounding like AI, because mostly I don't notice. But this was so over the top that I spent the whole article trying to decide whether it was an intentional parody of AI writing style.

This article's language is not en-US. It's not en-BR. It's en-SLOP.

Yes, that was my clumsy attempt at AI parody. Here's another: this article doesn't just have AI tells. It is AI tells.

Every sentence is saturated with AI style. Perhaps the author so AI-indoctrinated that they can't see this? It doesn't read as even vaguely plausible human writing. Which is mightily ironic given the thesis of "AI generated stuff is just fine, m'kay?" The writing style does more to defeat its conclusion than the analysis itself.

As for the substance of the analysis, it seems pretty good to me but I see some flaws that weaken it a bit.

The presence of "The Outlier Nobody Noticed" proves nothing and deserves no more than a passing mention. A random release introduced way more bugs than the Claude-containing releases. That provides evidence that Claude doesn't introduce more bugs only if your hypothesis is a very naive "AI is the only thing that can ever increase bug introduction rates."

The whole analysis has very limited data. It's necessarily based off a single pair of releases at the very end of the chronological timeline. You would never be able to reject a null hypothesis based only on that, so it's even less sound to present it as proving the null hypothesis. (By the same token, it would be incorrect for critics to claim that it proves their point. Did anyone claim this, though? The heated complaints seemed more based on priors about AI code.)

"The critics' claim is a simple comparison: did the rate go up?" That's reductive. For one, these releases are known to be in reaction to a flood of (AI-discovered!) security reports, which is a novel situation and in fact is a huge confound to anyone arguing about what those two releases mean -- they're both heavily AI-written, but in response to an unusual situation. When the samples are only drawn from a distinct scenario, statistic analysis can only speak to the quality of code in that scenario.

Also, another reasonable hypothesis could be: AI-written code has bugs of a different flavor that bothers users more. It's optimized for passing tests and convincing people and AIs that security holes are closed, which means other considerations like preserving functionality can more easily be regressed as compared to if humans were doing it. (If true, it still doesn't support the claim that depending on AI code is a catastrophe, fwiw.)

I'm not arguing the conclusion is wrong. I'm saying the analysis proves far less than it claims to. As for whether it's a debacle for rsync to become dependent on AI code generation, I think that's a reasonable debate to have but it's not going to be resolved this reductively.

The presence of "The Outlier Nobody Noticed" proves nothing and deserves no more than a passing mention. A random release introduced way more bugs than the Claude-containing releases. That provides evidence that Claude doesn't introduce more bugs only if your hypothesis is a very naive "AI is the only thing that can ever increase bug introduction rates."

It does not statistically prove anything, but as I thought I made extremely clear in the card where I discuss it, the point of bringing it up is different: to prove the hypocrisy of the anti-AI crowd.

By the same token, it would be incorrect for critics to claim that it proves their point. Did anyone claim this, though? The heated complaints seemed more based on priors about AI code.

The entire outrage is because people noticed what they thought was an unusual number of bugs and/or regressions in the release, saw it had Claude in it, and assumed a causal link, not just "priors about AI code."

You would never be able to reject a null hypothesis based only on that, so it's even less sound to present it as proving the null hypothesis.

The point I'm trying to make is that there is no evidence, based on these two releases, to think Claude made anything worse, whatsoever, and so the outrage is unfounded. This doesn't require me to prove Claude didn't cause any problems. If I ever made the latter claim, I should clean that up.

It's optimized for passing tests and convincing people and AIs that security holes are closed, which means other considerations like preserving functionality can more easily be regressed as compared to if humans were doing it.

Tridge actually explicitly says he made that tradeoff on purpose, not the AI.

Every sentence is saturated with AI style. Perhaps the author so AI-indoctrinated that they can't see this? It doesn't read as even vaguely plausible human writing. Which is mightily ironic given the thesis of "AI generated stuff is just fine, m'kay?" The writing style does more to defeat its conclusion than the analysis itself.

I've since rewritten nearly 100% of the prose in the analysis with my own, more inflammatory and verbose style. I also intentionally left in my natural mispellings and typos, to prove it was me.

This submission was heavily flagged, presumably because the article sounded like genai. But the article now says the following:

After posting this on Hacker News and recieving almost no substantive input, discussion, or response on the actual content of the article, I decided to rewrite all of the prose in my own voice.

I've therefore turned off the flags and hopefully people can actually now discuss the claims/findings being reported.

I decided to rewrite all of the prose in my own voice.

Soo... it didn't just sound like genai but was genai?

___

Huh. From the article:

If anyone complains about my verbosity or sentence structure — as they usually do, which is the reason I originally let the AI write the prose, among other reasons obsoleted by templating — they can go fuck themselves.

This is kinda sad, honestly. But also should show the author that doing what people try to bully you into doing will not stop them from bullying you.

Just stick with your unique voice man. If people don't want to read that that's fine. They do not have to. You're fine

.. what are those em-dashes doing there though?

Right so it’s gonna be a litmus test for knowledge workers going forward if they can separate style over substance. Genai tells are style. You have to be able to evaluate the ideas.

The em dashes are fine.

If someone gives them shit about their writing, that's on the critic for being shitty. If they use AI to write, that's on them for being fake. But, to write online at all requires being ready to have people be shitty to you and ideally not reacting in a way that makes the situation worse. Sounds like they need work on that part.

Anyway it is basically always possible for someone to find something legitimately bad about anything a person does. The question is, how much of an issue is that? Not much actually. So you have flaws. Fine, just be flawed. It had no affect on your life beyond your reaction to the attack. And putting aside that reaction is a prerequisite for learning anything useful (or discerning that there is nothing to learn) from the experience.

Good people will trust good intentions through the flaws, while shitty people will write off your work and your intentions because of the flaws (and try to make sure you feel bad about it in the process). But it's always they're too weak to express disagreement maturely, or sometimes because they're bitter and threatened by your good intentions directly. Either way, it's their flaw, not yours.

I don't think that you can successfully dismiss an obvious AI writing marker with

"No these are fine, now look over there!! <lotsoftext>"

Pay no attention to the man behind the curtain?

.. what are those em-dashes doing there though?

You're literally doing exactly the bullying I was trying to avoid, even while denouncing it. I like em-dashes. I have AuDHD, and they help me represent how I think.

So the original unfounded claim has 400+ comments because its perfect HN ragebait

The author provides evidence to the contrary and the HNers won't even engage with it instead just talking about the writing of the article in classic HN bikeshedding fashion.

How about after that we talk about the formatting of the website and the colors?

This site is really going down hill

Where is the accountability for your own opinions?

Are you guys only upvoting things that confirm your existing gripes?

Comments like this do more of what they complain about, only with an extra layer of judgment.

It would be preferable if someone would seed a better discussion by engaging with the article's claims/observations.

Why did you the admin allow such ragebait to stay on the front page then?

Is that the kind of low effort posts we want around here? Just a link to a github comment of a screenshot?

You're complicit here in fueling the harassment of an open source project

I don't have enough background info to understand what you're referring to here.

Even if you're right, though, you shouldn't be posting comments that break the site guidelines.

https://news.ycombinator.com/item?id=48342705

People opening issues just to rant against an open source project is acceptable content for HN?

How is that even allowed in the first place without getting flagged/removed?

And every time that happens the project gets brigaded from HN users

People have been ranting for, against, and about open source projects since open source began.

Please follow the site guidelines from now on.

I don't have a dog in this fight, but a few points that look a little suspicious:

- The release with the highest number of attributed bugs is the release _right before_ the first release with Claude-coauthored commits, released in January; is there a chance that unattributed LLM-authored commits made it into this release?

- The release attribution methodology is not great, since it will tend to attribute bugs introduced in a minor version update to the longest-lived patch release of that minor version. I doubt that 3.4.1 actually introduced a lot of bugs, but since it was released a day after 3.4.0, bugs that were introduced in that release get attributed to 3.4.1.

- Relatedly, more recent releases have had less time to have bugs filed against them, so there may be a bit of a bias toward evaluating recent releases as less buggy.

Your first and second points seem to contradict each other because if all of the bugs for 3.4.1 should be attributed to 3.4.0, that pushes the timetable back even further that unattributed LLM commits would have to have been being committed to the project, which just makes your point even more absurd.

Which brings me to my overall response, which is that there is absolutely no evidence, and nothing even intimating this hypothesis, that LLM commits were secretly being added to earlier releases before they were attributed, and that's why the rate of bugs is higher. There's no reason to think that it's an unreasonable thing to think, and there's no evidence for that whatsoever unless you beg the question and assume that higher bug counts must automatically indicate AI involvement, which is just circular reasoning. You're essentially just making up a hypothesis out of thin air to preserve your point.

Regarding your third point, that one's fair, but I've done the analysis and I can put it up if you want, as to how long it usually takes to find bugs and how far through the release cycle we are for each version.

Isn't the metric that you've used "bugs per commit ~ per new line of code" going to miss the issue?

All code is technical debt.

If rsync releases used to have 500 lines changed and 5 bugs in and AI-powered rsync releases have 50000 lines and 500 bugs, it's the same bugs/line but much worse experience for the user?

I've not looked into the details of this case and I do use AI assistance coding at work but in my experience, the problem is that it's too easy to write lots of code and therefore hard to review the huge volumes of code and this analysis will ignore that?

edit: actually your table shows there weren't unusually large numbers of commits in this release, so perhaps my initial skepticism shows a bias I have?

You can use LLMs in multiple ways, from very hands on to make local changes to completely hands-off.

I've seen plenty of code that was LLM generated but the commit message itself did not have the co-author attached to it. This only seems to happen when someone's interface to the codebase is completely though Claude/Codex/..., and those are usually the most verbose commits, and yet they say the least, because they just summarize the code changes, not the why.

On the other hand I've seen developers using Claude as a tool. They have VSCode open and a terminal window with Claude and go back and forth, ensuring they write correct code, and leave the plumbing to Claude.

So maybe the author of the code started off small and it grew over time?

I would expect a mature code base like rsync to have a lot of unit tests and integration tests and frankly if there's not enough that such bugs haven't been caught; that should be your first use of LLMs in order to setup some deterministic guidelines when you do start making changes to your actual code.

I have been experimenting with both aforementioned styles with interesting results.

Let's start with most outright alarming error - the claude statistics are taken out of whole 2 data points

That's sort of the point. There isn't enough data to extrapolate, and yet that's exactly what those outraged about AI were doing, and when you do do the very minimal types of analyses (permutation tests, and looking at distributions, mostly) that are actually valid, safe, standard, and useful to do on such low amounts of date, again, no evidence for the outrage shows up, and the two releases look so normal that it sort of shows no one would've cared if they hadn't known or found out that Claude was involved.

I really think this a much better standard of evidence — limited though it is — to outrage-fueled cherry-picked anecdotes, which is what has been driving this whole thing. If you disagree, and think the outrage should go one when I've shown there's an absence of evidence entirely for it (although of course, that's not evidence of absence; maybe I'll have to eat my words 5 releases down the line, but appealing to that now feels like a Russell's Teapot), would you care to explain why?

I know you’re defending your work here but this behavior does absolutely nothing to help your point.

The interpretations of the p-value is also alarming. One of the first thing they teach you in statistics class is: “an absence of evidence is not evidence of absence”.

This analysis showed that there is indeed an absence of evidence, but it concludes there is evidence of absence.

Traditional p-hacking is done by oversampling and overtesting. If you do 20 analysis on average one will show p < 0.05 by random chance. This analysis is doing the inverse of that. Under-sampling, and concluding with p > 0.05

This analysis showed that there is indeed an absence of evidence, but it concludes there is evidence of absence.

I tried pretty hard to avoid saying that, can you point me at how to rephrase? The point I'm trying to make is just that there is absolutely no evidence at all for what people are saying with such absolutism and claimed objectivity (that Claude made rsync worse), and thus it doesn't justify the outrage.

Under-sampling, and concluding with p > 0.05

How would I avoid under-sampling here? And if you're going to say it's because I only have 2 data points, well, the side making the positive claim — that Claude made rsync worse — only had two as well, and unremarkable ones at that, as I've tried very hard to show.

You are interpreting the p-values on their own merit rather then using them to test a null-hypothesis. Quotes like:

With a p-value of 74%, the answer is a decisive no. The odds ratio is 1.06 — essentially 1:1. Claude releases are no more likely to be above the median than any other releases.

are problematic in this context as the correct conclusion here is you just don‘t have enough data conclude whether or not you are more likely to encounter a bug after a Claude commit.

How would I avoid under-sampling here?

You don‘t. You admit that you don’t have enough data and move on. What you are trying to do here is prove a negative, which is extremely hard to do. In your discussion you claim that the users complaining had no right to, however nothing in your analysis showed they were wrong. We simply don‘t have enough data (yet) to say either way. When we have enough data they may be proven right or wrong, but until then, we cannot conclude either way.

If you insist still, I recommend looking into bayesian analysis. Theoretically at least the posterior distribution from a bayesian analysis can be interpreted directly and analyses on its own merits. However I suspect your posterior will have way too much uncertainty to reach any conclusions.

Edited that claim, and made several clarifications elsewhere. The whole point of this analysis is that outrage is unjustified on the basis of two totally statistically unremarkable releases that no one would have remarked on pre-AI (my further proof of this is that there was a pre-AI remarkably broken release, and no one did comment!) and zero positive evidence outside cherry-picked anecdotes for any negative impact. We should wait for outrage and version pinning and cancelation until there is evidence, no? I'm just trying to say that these specific releases are unremarkable, and there's no evidence at all of harm currently; I'm not trying to build any kind of predictive model for future Claude releases to say anything grander than "these specific releases are fine, what are we freaking out about?", not some claim about what Claude-exposed releases will look like or trend like in the future or in general.

There is a lot more context to the outrage which is missing from your analysis. People have multiple reasons to be mad at AI usage, you mention some of them in your introduction, and you put a (statistically insignificant) measure on only one of them. In your analysis you have shown that exactly one of these reasons is anecdotal. That does not mean they are wrong, and it especially does not mean they are unjustified.

That you found a single pre-AI release which did not cause outrage is proof of nothing. This single release is equally anecdotal, and statistically insignificant.

So, the biggest context that is missing here is that people hate AI for various reasons, and they don‘t want their favorite tools to fall victim to AI for equally many reasons. It is only natural that people who hate AI react this way when they find out their favorite tool uses AI, and doubly so when they sniff correlation between their favorite tools use of AI and bugs.

I'm just trying to say that these specific releases are unremarkable, and there's no evidence at all of harm currently.

Well, there is no evidence against harm either. But what you did here is a bit of a slight of hand. In your analysis your null hypothesis is: “There is no difference in bug count between releases which includes code commits from Claude Code and releases which don‘t”. (You then go about doing what every psychology major is taught not to do; find evidence for the null hypothesis, not against it). However what hypothesis testing is for is to use a representative sample to generalize over a wider population. You do hypothesis testing because you want to demonstrate that your sample is representative of a wider population, that you just so happened to have picked the two sample, by random chance, which shows the effect regardless of the experiment.

By calculating the p-values you were telling me that you were in fact ready to make generalizing statements over a wider population of commits, but your results were statically insignificant, so really you should not draw any conclusions from them. You have not, in fact, shown that they aren’t different from the rest of the population.

In your analysis you have shown that exactly one of these reasons is anecdotal.

This was actually the convincing one for me though. “Did AI increase the rsync bug rate? Dunno, can’t tell yet” seems like a fine conclusion to me. Plenty of people in this thread and previous ones on the topic seem convinced one way or another, so it’s nice to see actual numbers.

The concept you need here is "Statistical Power".

The ELI5 version is that there are two mistakes you can make when looking at a P value:

Type I error, where your P value is falsely low. In the experiment being discussed here, it would lead one to conclude that AI code is worse. Otherwise known as a false positive.

Type II error, where your P value is falsely high, leading you to conclude that AI code is no different. Otherwise known as a false negative.

https://en.wikipedia.org/wiki/Power_(statistics)

One can calculate statistical power for a given experimental protocol.

My hunch is that if you did this, you would find this experiment is grossly under-powered.

This means you can't make the "absence of evidence" claim.

He can't make the evidence of absence claim, but he can absolutely make the absence of evidence claim.

Perhaps in an “everyday language” way, but not in the technical, statistical sense.

In an underpowered statistical study, a claim that two experimental conditions did not differ are not persuasive.

No. It's a description of the result of the maybe underpowered study. the underpowered study did not find evidence. Evidence is absent. Because it is underpowered, it's not evidence that the effect is absent.

The claim is not "two experimental conditions did not differ". The claim is "The data do not show evidence that the experimental conditions did differ".

You say "the underpowered study did not find evidence". Not true, it found quite a bit of evidence - many statistics were presented. There is no absence of evidence. The author wrote about the evidence, presenting P values and other statistics.

Of course the critical part is not the numbers, but what they mean.

So, what does the evidence mean?

The author interprets it to mean that there is no difference. They state this several times:

"46% EXACT PERMUTATION TEST P-VALUE (ONE-SIDED, H₁: CLAUDE MEAN > HISTORICAL)[...] What this p-value tells us is There's nothing unusual about the Claude group."

"74% ONE-SIDED P-VALUE (H₁: CLAUDE MORE LIKELY ABOVE MEDIAN) Fisher's exact test asks: if we split all releases at the historical median (0.74 sev/10c), are these Claude releases significantly buggy than previous releases (more likely to land above the median)? With a p-value of 74%, the answer is a decisive no. "

In an under-powered study, when a P value is above your alpha level cutoff (.05, .01, whatever was chosen) you can't distinguish between "no effect" and "could be an effect, but I didn't see one".

Agree. From the article:

Here's my favorite part, though. Digging into the data, one of the first things that jumped out at me with blinding clarity was that the worst release, by far, in rsync history was entirely prior to the introduction of Claude ... And yet nobody noticed.

Language really does suggest the article's author does have a dog in this fight and is cloaking opinion in fancy statistics jargon. "Blinding clarity"? All you have to do is draw a plot. And anyway, v3.4.1 was 2025-01-16, technically well within the AI assisted coding era and before attribution was becoming standard practice.

Also from the article:

"Claude clearly made things worse" &emdash; the main claim

This article was clearly generated by AI, yet I found no mention/attribution of that by author.

How likely is it than someone who vibe codes articles would also vibe code the underlying analysis and be eager to accept an outcome that is highly validating of that person’s workflow? I’d say very.

Are the numbers wrong? That's the only relevant thing here.

Also, humans do use em dashes, just FYI.

He did admit as much:

"The scripts used to fetch the data, collate it into a DuckDB database file, construct the views on that DB, and then do the statistical analysis on that data, were indeed written by GLM 5.1, as was the HTML and much of the original prose for the final report webpage you're looking at right now."

This is a neat post and I'm glad it got written and this is a little bit off-topic but:

Hey, 'logicprog, your writing is fine!

Use LLMs to critique your writing, check its structure, vet your choice of topic sentences, check flow from graf to graf and section to section, look for passive voice and overused words. LLMs are fantastic for that. But don't use a single word an LLM suggests in your actual writing. If it suggests something really fucking good, too bad, those words are disqualified. It's an easy red line to adhere to, easier than it sounds, and it'll keep your writing human.

(You ended up somewhere around here anyways, but that was after you posted something with LLM-written language because you weren't confident enough in your own writing. The things you do "worse" than an LLM are what make you you; be protective of them!)

needs a tldr; im not reading all that. maybe claude can summarize it for me.

And anti-AI people accuse people who use AI of being intellectually lazy. First of all, it's long because it's expanded to respond to all the criticisms. It seems that either something can be short, and dismissed as incomplete, or it can be complete, and dismissed as being long. Nice Kafka trap. Additionally, there's literally an Executive Summary section right there, for your TLDR.

The fact last few commits were attributed to claude doesn't mean previous ones didn't use it.

Also if you write a paper where you get statistical conclusions out of whole 2 datapoints you'd be laughed out of the room

Tl;dr:

Yes, it did. Here is some math showing that you shouldn’t care about that.

In what way did it create more bugs? It literally doesn't show up in the data. What are you talking about?

The only reason why people are talking about this is because of the bugs in the code that the chat bot generated OP. People updated to a version of rsync that didn’t work right, the one with all the bot commits in it. This blog post is about how claude didn’t create more bugs than usual if you think about it in one very specific way, not that it didn’t create more bugs at all.

It is like if your neighbor opens your door and a dog walks in, there’s no point in doing some weird analysis about all the times you yourself have let a dog walk in. He still did that.

That doesn't make any sense, what?

I am not sure what to tell you here, because somebody literally posted the code of a bug that Claude inserted here

https://news.ycombinator.com/item?id=48419197

And your response to someone pointing out that sloppy, buggy code that Claude introduced, was to just quote Tridge (which does not in any way refute the fact that you’re looking at a bug that Claude introduced to the code)

https://news.ycombinator.com/item?id=48419621

I’m not entirely sure what the purpose of this project is (maybe to “prove” Tridge’s opinions about LLMs and human intelligence that he made in the linked blog post to be right?), but it appears as though you are ignoring irrefutably true observations. You just asserted that “the data” doesn’t show Claude introducing any bugs (which is a bizarre claim) after previously responding to a documented bug with a… deferral? Do bugs not count if you can find a vague excuse for it?

There is nothing in the blog post that is evidence that Claude didn’t introduce bugs. It is a thought experiment that uses “increase bugs” and “increase bugs more than a given arbitrary statical amount that I selected” as interchangeable statements.

This is just narrow thinking. Say Claude did increase the bugs in rsync by a negligible factor.

So what? You've saved a significant amount of time for a decent number of humans, and if those humans are working on other projects, the overall net output for the world is net positive compared to without LLMs.

You have to broaden your perspective. It's not just about how rsync was affected.

Wait, how is any of this relevant if there were only 2 Claude commits? My statistics courses are far behind me, but don't you need at least 30 data points to conclude anything?

How does the analysis look if we only count the >=90 severity cases—that is, if we downgrade the severity of all <90 cases to 0?

Write with your own voice and then polish with ai.

It's always the most insufferable people that make the biggest hullabaloo about a project they have nothing to do with and have never contributed to. People with literally zero skin in the game using the AI boogeyman to push some agenda or some anti-agenda. OSS has become so incredibly toxic in the past decade, and consumers of OSS have become extremely entitled.

I run a smallish project with ~1k stars and I've stopped maintaining it last year because people feel like they're absolutely owed features or bug-fixes or whatever. It's tiring and a complete shame that author has to make such an insane deep dive into a random accusation that just caught on social media. I want to emphasize that this has nothing to do with AI, it's just tech tourists, consumers (as opposed to creators), and engagement farmers that have taken over. AI slop probably doesn't help, but the underlying issue has been brewing for at least a decade.

Also, the "making soup for the homeless & pissing in it" is not only an off-base analogy (software is pretty low on Maslow’s Hierarchy of Needs), but also somehow looks down on both people in need and the volunteers that help them. Just absolutely gross.

Was just looking at commits and came across a commit and its revert

original commit: https://github.com/RsyncProject/rsync/commit/d046525de39315d...

```

- if (!ptr)

- ptr = malloc(num * size);

- else if (ptr == do_calloc)

+ if (!ptr || ptr == do_calloc)

   ptr = calloc(num, size);

```

Written with claude. This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a strict upgrade. For large and recursive allocations, this becomes a significant cost.

reverted in https://github.com/RsyncProject/rsync/commit/7db73ad9a1b8721...

if you read the description of revert half carefully, it's easy to tell that even that was written by an LLM .

I can understand the sentiment of whoever posted the original thread.

Also the amount of commits is suspicious. In the last two months, rsync had about as much commits as in the last two years before that. Most of them written with claude. And then stuff like this is in there.

That's exactly what I'd expect when someone is excited about AI usage and becomes... well, sloppy.

Tridge already explains this:

"Like many developers of open source packages I’ve been hit by a flood of security reports lately in my role as the rsync maintainer. Many of those reports are AI generated (not all though, there are some notable ones with very careful and high quality manual analysis).

As this flood started to get more intense I realised I needed to raise the defences on rsync a lot — we needed much more thorough test suites, code coverage analysis, CI testing on a lot more platforms, deliberate and thorough scanning for possible security issues (so I find at least some of them before other people!) and the addition of a whole lot of defence-in-depth hardening techniques. This is all a huge amount of work. "

https://medium.com/@tridge60/rsync-and-outrage-d9849599e5a0

I think Tridge is simultaneously trying to be proactive and kinda giving too much credit to marketing. Anthropic has not been able to really give numbers or actual values on what Mythos can really do. It just waved Mythos in front of the public like a boogeyman screaming that AI is going to cause a security nightmare (and it has, but mostly through vibe coded trash from what I’ve noticed); I’m hard pressed to find their statement that they spent less than $20,000 to find a Kerberos bug in FreeBSD a compelling win without a lot more context and they seem disinclined to provide that data. I really do wonder what evidence they have provided to their approved partners, all of this smells…weird.

I honestly think the main problem is Tridge just failed at communicating any of this correctly and I don’t think the implication he gives that all of this was due to the urgency of the impending security apocalypse really holds water.

Why was all of this written straight to the master branch? Now that the release is out, why not better explain what the urgency of this release was? Why wasn’t he proactive in communicating this and instead let the mob make up their own story? I think a lot of people are inclined to give Tridge a lot of leeway due to the fact that he literally is the reason why rsync exists, but this was avoidable and I think the comment in his response post where he mentions that, “I’d rather be out sailing than working on rsync security issues, so I have reached for several AI tools to help with what needs to be done,” speaks volumes as to what is going on.

As a long-time open-source maintainer, I find all the second-guessing and armchair psychoanalysis here (not just in this comment, all over HN) about Tridge's motivations, state of mind, and so on incredibly off-putting.

Tridge doesn't owe anyone anything as far as rsync is concerned. Yet he is spending his time maintaining it, only to be attacked for his efforts.

To respond to the specific technical point, there really _is_ a flood of security reports arriving everywhere in the past few months. The jury is out on whether Mythos is that much better than alternatives, but even the publicly available models are _highly_ capable of finding real problems, and they are being employed to that end quite effectively. Here are the counts of security issues fixed in each monthly Go minor release going back to the start of 2024:

     0 2024-01-09 Go 1.21.6, Go 1.20.13
     0 2024-02-06 Go 1.21.7, Go 1.20.14
     5 2024-03-05 Go 1.22.1, Go 1.21.8
     1 2024-04-03 Go 1.22.2, Go 1.21.9
     2 2024-05-07 Go 1.22.3, Go 1.21.10
     2 2024-06-04 Go 1.22.4, Go 1.21.11
     1 2024-07-02 Go 1.22.5, Go 1.21.12
     0 2024-08-06 Go 1.22.6, Go 1.21.13
     3 2024-09-05 Go 1.23.1, Go 1.22.7
     0 2024-10-01 Go 1.23.2, Go 1.22.8
     0 2024-11-06 Go 1.23.3, Go 1.22.9
     0 2024-12-03 Go 1.23.4, Go 1.22.10
     
     2 2025-01-16 Go 1.23.5, Go 1.22.11
     1 2025-02-04 Go 1.23.6, Go 1.22.12
     1 2025-03-04 Go 1.24.1, Go 1.23.7
     1 2025-04-01 Go 1.24.2, Go 1.23.8
     1 2025-05-06 Go 1.24.3, Go 1.23.9
     3 2025-06-05 Go 1.24.4, Go 1.23.10
     1 2025-07-08 Go 1.24.5, Go 1.23.11
     2 2025-08-06 Go 1.24.6, Go 1.23.12
     1 2025-09-03 Go 1.25.1, Go 1.24.7
    10 2025-10-07 Go 1.25.2, Go 1.24.8
     * 2025-10-13 Go 1.25.3, Go 1.24.9
     0 2025-11-05 Go 1.25.4, Go 1.24.10
     2 2025-12-02 Go 1.25.5, Go 1.24.11
    
     6 2026-01-15 Go 1.25.6, Go 1.24.12
     2 2026-02-04 Go 1.25.7, Go 1.24.13
     5 2026-03-05 Go 1.26.1, Go 1.25.8
    10 2026-04-07 Go 1.26.2, Go 1.25.9
    11 2026-05-07 Go 1.26.3, Go 1.25.10
     3 2026-06-02 Go 1.26.4, Go 1.25.11

* The Go 1.25.3 and Go 1.24.9 releases were a fast follow to fix a problem introduced by one of the security fixes the previous week.

You can see that 2026 has been quite different from the previous years. There are plenty of other contemporaneous accounts from other security teams about the load increase they've seen (which again is almost entirely not Mythos).

Also, the number of reports we are receiving has gone up far faster than the number of actual vulnerabilities. Over the 75-month period from January 2020 to early April 2026, the final 30 days accounted for ~16% of the reports.

It is easy to believe that Tridge is seeing a similar flood of reports. More reports means more fixes means more code changes means more bugs.

As a long-time open-source maintainer, I find all the second-guessing and armchair psychoanalysis here (not just in this comment, all over HN) about Tridge's motivations, state of mind, and so on incredibly off-putting.

I agree that the entire episode is obscene, but I am also unsure of what to do here either. On some level this is the same problem movie stars run into. I agree that guessing or waxing about the motivations of anyone is a nosy and overall unproductive exercise (yet paparazzi exist because of this very human behavior), but I also think that there is a modest duty owed to users to explain things.

Tridge doesn't owe anyone anything as far as rsync is concerned. Yet he is spending his time maintaining it, only to be attacked for his efforts.

I am reminded of this piece: https://mikemcquaid.com/open-source-maintainers-owe-you-noth...

Which, I empathize with, but I fundamentally disagree that maintainers owe users nothing. I will die on that hill. If you are getting to that point where you actively loathe working on the project, I agree you should be able to walk away. However, I strongly believe that when you create something for people to use that there’s an implicit social contract about how to go about doing certain things.

I suppose in a very extreme and intentionally histrionic example, having a project carry the MIT license, getting frustrated and then changing the project to delete the entire system is a crime. The average person and the courts don’t care if the license is “as-is”. There is a duty that is understood that you don’t do that and I think we need to make it clear what that duty is for OSS.

Ultimately, though, I think this is all symptomatic of the fact that the OSS model has gaps that the increase in security reports whether AI generated or not has exerted more pressure on. I have certainly been on the receiving end of a lot of frivolous security reports that were discarded because it was obvious that it was just someone with a security scanner wandering around the Internet. You still have to review that nonsense and it eats into your time. Doing this on your own time, without pay and having to listen to the peanut gallery is just infuriating.

Is any business built on top of rsync going to donate their money in a sustainable manner?

Yeah the maintainers don’t owe users nothing is a disgusting sentiment that doesn’t stand real scrutiny. There is a social contract here. If you want to be respected and get recognized as “tridge” or whatever your name is, you owe the people that recognize you and that wider community in general.

No there isn't.

First off: I don't agree that there's a social contract here at all. That's just some imaginary thing that you (and others) have decided exists. It's funny how lots of people who aren't open source maintainers seem to think it's ok to make up social contracts for other people without their consent.

But ok, let's just pretend for a second that maintainers have indeed entered into some sort of social contract that gives them an obligation to support their software, uncompensated. But if we have this contract, then it cuts both ways. The users then have entered into a social contract of their own: they agree to treat me with respect when they deal with me, to not act entitled, to not demand things of me, to not be rude, and to do their part in being a helpful, productive partner in helping to solve any issues they report.

If a user breaks their part of the contract, then I have no obligation to fulfill my side of it.

It's a bit bizarre to me that non-maintainers have decided to invent some sort of "social contract" that benefits them (while putting a sizeable burden on maintainers), but seem to think that they aren't entering into a social contract of their own when they decide to use the software. (And that there are consequences for not upholding the user side of the social contract.)

Put another way: in contract law (in the US, at least), there's the concept of "consideration". It's the idea that both parties are getting something out of a deal. Some of that can be monetary, but it can also be other things. If a contract is one sided, that is, if one party isn't getting any consideration, then the contract can often be unenforceable.

That seems to be what people like you are doing here: requiring that open source maintainers enter into a social contract, but not give them any consideration in return for it. (And no, some sort of ill-defined concepts like "reputation" or "large user base" don't pass my threshold for meaningful consideration.)

That's one more thing, even: contracts are voluntary. All involved parties must agree for there to be a contract. I don't agree to your bullshit contract of one-sided obligation, so there is no contract.

I agree, it's very off-putting, and I totally understand that the amount of reports are overwhelming for maintainers of popular libraries.

More reports means more fixes means more code changes means more bugs.

Sounds like we'll be riding a downward spiral for the foreseeable future? It will be very interesting to see how stats like the ones you shared develop in the coming year(s).

From the article I find this a bit concerning:

So: the Claude releases changed way more lines of code than historical ones, but didn't have more bugs. More code, same bugs. That's not what you'd expect if Claude were making things worse.

More code, same bugs, is a net negative, no? I mean unless it's strictly needed for the inherent complexity of the program. But I've seen a tokenizer written by Rob Pike and I've seen a tokenizer written by Claude.... they are not the same :D

As a long-time open-source maintainer, I find all the second-guessing and armchair psychoanalysis here (not just in this comment, all over HN) about Tridge's motivations, state of mind, and so on incredibly off-putting.

Much of the language from both groups is incredibly off-putting, frankly. Tridge in his blog post describes people as "foaming at the mouth"?!

The rhetoric around this has gotten way too emotional from both groups.

I'm glad I'm just a hobbyist.

IMHO, the whole episode is just embarrassing. I have no doubt he’s just trying to do the right thing. You can disagree with the tactics, but the vitriol is outrageous. rsync is a gift to the world and we should be grateful and mindful of how much it has been quietly woven into the fabric of computing. rsync is taken for granted. This is not okay.

This is not okay.

Agreed. The way to address it though, is through calm analysis and reason. The emotional language from both groups is not helping.

If there's one problem with Claude et al, it's that it's all happened way too quickly for people to keep up. We're all at different stages of acceptance and I think that's what we're seeing manifest in the various discussions.

Yet he is spending his time maintaining it, only to be attacked for his efforts.

Which, in general, is totally legit. Doing something voluntarily doesn't relieve you from criticism if what you are doing isn't good.

“I’d rather be out sailing than working on rsync security issues, so I have reached for several AI tools to help with what needs to be done,”

Well, then maybe it's already overdue to find a new maintainer for the project and let someone else continue it? The tool will not get better from someone working on it who doesn't want to.

Unless you're willing to step up and be that person, it's not your place for you to suggest it.

I don't agree with that, I can very well still discuss that. He clearly sounds like someone who doesn't want to do this work anymore and should have searched for a successor.

That's my impression from that sentence, at least. Don't you agree?

So, why didn't he do it? Because just firing up Claude and let it rip is way easier than finding real people and building up trust?

Did Claude increase bugs in rsync? Or did Claude just gave some basically retired programmer, who doesn't even want to work on his project anymore, the impression that he can replace finding a successor with just handing it to AI?

and should have searched for a successor.

He doesn't have to do that. If he ever do not care enough he can just stop maintaining it and that's it.

Is using calloc for everything fixing a security issue or hardening it?

Calloc is generally hardening, because it zeros out any stale memory contents left over from previous uses of the memory.

You can avoid this overhead if you use a language that forbids reading from uninitialized memory, but C is not that language.

Uninitialized memory is not a problem (the OS is never going to give a program memory that has data in it from another program). The problem is memory that you allocated in the past, have freed, but hasn't been returned to the OS[0]. It might have key material or other sensitive data in it[1]. Or it might just have random garbage in it that could be misinterpreted by the code that's about to use it, if it hasn't been initialized to a known state.

For some uses, you do genuinely need (specifically) zeroed-out memory before you start to use it, and that's where calloc() is truly useful. But that need not have anything to do with security.

[0] The allocator will often hold onto memory that has been freed in order to quickly service future requests for new allocations, without needing a context switch into kernel space.

[1] Granted, the correct way to handle that is to zero it out before freeing it, in a way that the compiler won't optimize out.

Seems like someone could use Claude to port rsync to Rust and the whole enterprise would be safer from things like this.

Start with unsafe then gradually convert into idiomatic Rust.

Your let's redo this in Rust made me wonder if generative AI will also be susceptible to software fads. One LLM writes a few blog posts extoling a new framework/lanaguge. Other agentics read these and get 'influenced'. Then they start clamoring for 'lets redo this in X!'. Can't wait to see it. /g

You can get 80% there with rust which is what is impressive. Then you have a reference implementation that you can always check against. If a Rust library have 0 unsafe, i dont care if it is written by a dog, it still have 0 UB.

This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a strict upgrade.

I wouldn't assume Claude made that decision; it's not as if that was some incidental thing that it snuck into a large commit. The commit message starts with "zero all new memory from allocations", and that's exactly what the commit does. What do you imagine the prompt was?

It seems totally plausible to me that a human initially thought this was an improvement, then rethought after discovering the RSS regression. And it's not a law of nature anyway that this change has to increase RSS; calloc could special-case the case in which memory was freshly returned from the OS, knowing fresh memory mappings are zeroed anyway.

I blame AI for these regressions mostly in the sense that it caused a flurry of vulnerability reports. Those led to a flurry of quick fixes. Sometimes quick fixes cause other problems.

You don't really have to guess. The guy told us the AI didn't suggest this specific change:

The change to zero memory was my idea and my change. It was a reaction to a security report I got which caused use of an element past the end of an array. By zeroing the allocation I could ensure that misuse of that memory if a similar bug came up in the future could only cause a null ptr deref, which is better than the chance of a valid pointer. It got a claude co-authored tag on it as I got it to do some tidy ups of a series of commits, and that is just what it does when it makes any modification. It doesn't mean the change was written by claude. It was written by me.

https://github.com/RsyncProject/rsync/issues/959#issuecommen...

okay I had not read this or any discussions there (except the one linked in the post), but this looks weirder. the comment you linked is a dev responding to what is very clearly a bot comment. I am sure they have good intentions and I have no reason to believe otherwise as I have no connection to the project whatsoever, but the original commit being 4-5 lines long (what did claude do then?) and the revert description is almost certainly written by an LLM makes in my mind the slop argument stronger.

I hope if this doesn't come across as unkind towards the dev who gives their time and energy to the project. Grateful for that.

… By zeroing the allocation …

How does that prevent reading past the end of the buffer? Or change how bytes outside the buffer are used? Are these arrays of pointers so that the “null ptr deref” comment makes sense?

Or am I the bozo and don’t know what’s happening here?

It doesn’t. It’s just that dereferencing a zeroed pointer reliably crashes the program (unless you specifically do funky things with mmap) but dereferencing garbage memory as a pointer could do a lot more insidious damage.

My point is that the developer's comment doesn't make sense. Zeroing the allocated memory doesn't change anything about overrunning the buffer.

edit: removed unnecessary examples

AI multiplied by Linux overcommit. What times we live in!

(My own view: 10.8 GB is nothing these days. Your sprintf buffers are probably larger than that. (And if they aren't: they should be. That, or you should start using snprintf...))

sprintf() should be a longer way to write abort(), change my mind

I'll change your mind:

If you pass NULL as the destination pointer, it doesn't write any string. If you combine this with %n at the end of the format string, you can get the exact length that the output string would be. Then you allocate that, then you print again, into the actual destination buffer this time.

If anything I just got entrenched in my opinion. The second best option, maybe, would be to not accept any destination pointers and have the default and only possible behavior just like what you describe.