Ironically this seems like a perfect use for AI from the maintaner side. "Find low-effort/AI-like PR's without associated issues from new contributors. Discuss with the contributors in the PR about how contributions should be made, then reject their PR's. If any bad behavior is detected in this interaction, block the accounts for 24 hours and add them to a list of accounts to review for permanent banning from the project".
Automated PR's can have automated responses. You make a human effort, you get a human response.
We do similar in Homebrew by autoclosing opens without the issue template (which implies using the API which often now implies AI).
The weird flip side is that the average good AI contribution is better than the average “I used no AI” contribution. Perhaps it’s because Homebrew has so many declarative guardrails and is so easy for agents to test.
I think the idea of leveraging AI to find useless PRs is right.
I didn’t like the second part though where it starts banning people for “bad behavior”.
I get that AI resistance will be a thing in the next few years, but realistically engineers who plan on being around for the next 20-30 years just need to embrace it instead of being sour about it.
Because of machines, things have arguably improved for humanity as a whole. People who seek more elevated/finer products can still turn to handcrafted products.
Mass software will be produced by AI, and it will scale just fine.
> Because of machines, things have arguably improved for humanity as a whole.
That's a very narrow view of the situation. I agree that this side of the coin exists, but there's also the other side where drinkable tap water is becoming a luxury in much of the global North, giant tornadoes and fires are sweeping the Earth, and temperatures are growing out of control.
So, to make lives more comfortable with machines for some millions of privileged people, they triggered the 6th mass extinction. That's the other side of the coin.
I just think when you close PR's outright, you tend to get angry responses. And as a maintainer, you don't want to deal with angry responses. People who post bot PR's but can't accept having their PR's closed by a bot, shouldn't be visible to the maintainers.
Which part of the second part do you dislike more?
That AI will be given the power to ban people?
Or that AI will be the ones getting banned?
If it's the former, expect an invite from the AI resistance. :)
If it's the latter, well, that can be fixed by spending more tokens and writing better PRs so that the AI moderator doesn't think they're "low-effort, AI-like". I've seen my share of low-effort PRs from humans, too. Low-effort is low-effort, whether it's from humans or AI.
> I didn’t like the second part though where it starts banning people for “bad behavior”.
I think you failed to notice the ban would be a reaction to recurrent bad behavior, such as ignoring maintainer requests.
What's your solution to handle accounts which spam projects with nonsense PRs?
> get that AI resistance will be a thing in the next few years (...)
This is a very lazy opinion to express. If you pay attention to the post you're replying to, you will notice that the problem isn't AI. The problem is spamming projects with PRs that are meaningless while completely ignoring maintainers.
You'd have the same problem if some guy decided to spam the project with a constant barrage of small low-effort nonfunctional PRs.
As a contributor who had to deal with 2 automated reviews in the past month, it's a bad experience. I really wanted to upstream my changes, but these bots made me question if i should at all.
Bots don't understand how their words can hurt people's feelings, and can even less understand that talking to a LLM is even more sad/infuriating for actual humans. I can handle CI/tests just fine, but LLMs playing on my human emotions will have me swearing in no time.
I contribute to free software projects because i want user control over the machine (that was the idea, long before LLMs). Slopware might technically be FLOSS, but it's certainly not in the spirit of FLOSS.
While exploring an open-source project recently, I noticed a bug in the initial setup. I asked Claude to verify it, and the AI confirmed the issue. When Claude asked if I wanted to file a bug report, I agreed, assuming it would simply generate a formatted text block for me to copy and paste. However, I soon realized it had actually submitted the issue directly to GitHub. It turned out I had the GitHub CLI installed and was logged in with my work account.
Out of curiosity, I then asked Claude to draft an enhancement request for a new feature. It created a highly detailed issue and went on to generate the corresponding code for the enhancement.
I haven't submitted a pull request because the code is written in a language I don't know. Part of me considered submitting it anyway since it works and could be useful, but after reading this post, I think I'll just keep it under wraps.
IMHO, the days when open source contributions are often positive signal for hiring are long gone.
Actually, if I see someone doing open source like it's a performative career checkbox, that will not be positive signal, and could easily be negative. I understand that people will do what they need to do to get a job, but that pragmatic career checkboxing itself isn't positive. I will have to look for positive signal elsewhere for that person.
The problem is that our industry has gotten very bad at hiring, and now we have everyone playing all sorts of games with it -- rehearsing Leetcode interviews, bad faith open source contributions, feigning enthusiasm, spamming AI-tweaked resumes, outright cheating on interviews -- rather than focusing on doing good work, and being part of a team.
If you're involved in hiring, and you care about effectiveness and culture, consider pushing back against the prevailing dysfunctional big-corporate-cattle-herding practices. Especially if your company has no excuse to be big-corporate dysfunctional, and can't afford to be.
Doing good work leads you nowhere. You need to be in the game of power. Be visible, know every 3 levels of managers by name. Otherwise you’re only number in payroll’s sheet. Modern tech companies are not different from world war 2 era factories imho. Account xyz@bigcompany pushing lines of code to endless repo. No face, no name. Boring. Boring reasonable and reliable. That’s not what hiring people are looking for. You must be a rockstar, conference speaker and leader of few open source projects now. Skills, experience and good work ethic is not enough.
Open source contributions as a signal of developer quality is a thing of the past.
Open source contribution on github specifically is no longer about building and maintaining something, its a parasocial signal that has more in common with linkedin hustle bloggers.
So the fixes are still fixes, but we (I am also a OSS maintainer) are unwilling to accept them as they boost the contributor’s status where we think the merit is very or extremely limited.
Why not have these PRs counted differently (by the platform), and/or colored differently in the timeline(s) thus made less visible or more clear?
Unless the change is an obvious improvement, it has to be worth the time for the maintainers to spend attention reviewing it (and supporting the code forever, and all the rest).
Even if these particular changes are "harmless" and easy to review, accepting them sets a precedent that encourages an unsustainable flood of AI-generated changes that will overwhelm the project.
That’s a totally unnecessary high bar you’re setting. Most change is mediocre (a tautology, I realize). I really don’t believe that you can think that fixing a bunch of spelling mistakes is not a small improvement.
About setting a precedent: if you think like that you probably never accept any contributions at all, by humans or otherwise. Contributions by strangers have always been very minor things in general, things that the author cared enough about to make a pull request. In this case I don’t see the difference except for the fact that ai was used. If you just don’t accept ai at all , fine. But just be clear about it. In this case they are calling spelling correction slop. That’s not what slop means. Pretending human contributions were always “great” and nothing else would do is kind of ridiculous and shows a lack of experience in open source.
Change is bad. It takes engineers time to understand and approve. It confuses users. Changes should be made for good reasons, not frivolously.
Just because writing code is easier does not mean you should write more of it. It is more important than ever to carefully consider what you work on and what work you accept.
The user says "this is just spelling corrections" but a maintainer - a volunteer - needs to read all the files and make sure it's harmless and doesn't introduce security vulnerabilities, change meanings, and so on. That's not free.
These fixes, regularly destroy the architecture, accrue bloat for little gain, refuse to rewrite while demanding to rewrite- and many other such funny noises. Most code contribution by LLMs is garbage if you long-term care about the project. Look at closed source projects that ingest all this madness - windows with its seconds to open the explorer and other catastrophes.
I think Goodhart's law applies really well here. When people make commits for the sake of having made a commit to some famous repo, the "merit" that a commit measures is already useless. Remember Hacktoberfest 2020, before LLMs were popular? https://news.ycombinator.com/item?id=24643894
What merit was there to have anyway. Would the author accept the same changes from a human?? I am inclined to believe they would, I certainly would. Would you have attributed merit to someone just fixing typos? I mean it’s quite helpful. But I wouldn’t think a commit fixing typos ever attributed merit to a developer. So it’s hard to understand the problem with AI doing this. If it was a bot like dependabot but for fixing grammar errors would that be a problem? I assume no, hence the problem is entirely that the human author of the PR is being attributed nonexistent merit by people who should know better.
Well, as I mentioned there was already in 2020 the problem of (human-generated) slop commits causing unnecessary work for maintainers.
The commit in itself might add some minuscule value to the project, but when you take into account the time it takes to check the commit and approve it, you're probably back at negative value again.
Plus, there's the problem of people doing this in an adversarial way. How can you tell whether a random commit touching 100 files is the work of an enthusiastic beginner, or a malicious hacker that's hidden a subtle attack somewhere? Well, you need to spend your time carefully checking it.
So it's way more pragmatic just to set up some automated filter to put all of this stuff in the bin.
Oh and by the way:
> I wouldn’t think a commit fixing typos ever attributed merit to a developer.
The first sentences of the article in question:
> Successful contributions to open source projects are a kind of currency. GitHub in particular encourages this in a number of ways: [...] Potential hiring managers often take note of this. Recruiters often find and screen candidates this way.
Because letting sloperators near your codebase, even for a very minor obvious and correct change, may have long-term negative consequences for your project. It sends a signal to other sloperators to keep doing what they do, when we really need to send them a signal to get a life, learn a craft, and let Claude burn in hell.
Because every merge request and every merge comes with cost and (potential) more cost in the future.
Linux lately received a lot of fixes for drivers nobody has cared about for ages. They decided to remove them. Maybe they should have done this earlier, I don't know, but that happened because every code change comes with cost down the line. For real contributers, this cost is much more limited and they have a higher probability of burden the cost down the line.
Pure AI MRs are just rude, because it's just "here take this, don't really care what this is, but AI said it being good, I won't be around, so you will have to deal with this code after the merge and you better don't overlook anything, because I personally don't really know what I am doing, I only know how to have AI making something that looks convincing. Good luck fixing the bugs this change introduces".
My solution is to ask them for proof of work. Many of these disappear after opening a PR that they never tested. How can you claim to have fixed a visual issue without a screenshot? Insta-close.
I have noticed a lot of my Github followers who are either newer students or just new to Github have an immaculate green board of commits on their profile page, but it's so over the top I only like the ones that have creative illustrations or just very limited edits. Some seem so automated that it's hard to know how much they actually work on it. Others even are programmed to follow accounts then drop off once the person doesn't reciprocate, which is technically better than them staying but also systematically checking what sticks (also Github should limit some of these automated follows).
Not just that how the maintainer was treated by the attacker which tells you something about the way people feel entitled to maintainers time and efforts for nothing even by people not looking to exploit the project. And the abuse after he received despite it being an easy mistake that the culture encourages people to make.
> And the abuse after he received despite it being an easy mistake that the culture encourages people to make.
I believe that that was in part simply people defending their reality.
Because if it's not the maintainers fault, then why did it fail? What are the implications of that? What does it mean for me personally? Do I have to do something differently? Do I have to change my assumptions?
People do not like these feelings. They want to simply push them away.
The easiest way of doing so is blaming that guy and being done with the negative emotion.
Most people do suck at this whole "being human" business.
I think what you're saying is fair, tbh. Imagine you put in months or years into a project to make it a quality piece of work. It develops into something notable and you took all the risk. Then someone comes along to fix a spelling error with a pull request so their name effectively appears on the repo as a "contributor." And you just know right after its going on their resume as "contributed to [...]" or maybe if they're bold "software engineer working on [...]" which implies substantial investment. Then you're effectively sharing credit for YOUR work with someone who did nothing. That is rage inducing. ((Of course: it probably is just juniors trying their best in this horrible industry.))
On the other hand: lets be careful not to dismiss valid but inexperienced attempts to contribute. Having someone want to genuinely contribute to your software is incredibly generous. If someone seems like they're trying its better to give advice than act like a snob because its not good enough. Often pull requests only need small fixes to get in, anyway.
Hi Neil, fun to see you on HN. I agree with your points and I you summarized it very well as "Ultimately, open source is built on trust".
AI is destroying trust in open source and many other areas and I think this will discourage teams from publishing their source code in the future.
On the other hand, personal connections are becoming even more important, which is unfair to the younger generation and people who don't live near tech hubs.
Today's conception of open source is different from the original anyway. It used to be "we made this thing, here's the code" but now it's turned into some kind of weird parasocial thing more akin to Facebook.
> Today's conception of open source is different from the original anyway. It used to be "we made this thing, here's the code" but now it's turned into some kind of weird parasocial thing more akin to Facebook.
I didn't agree with your comment when I glanced through it, but then further ahead in the discussion I stumbled upon a whole thread of people commenting that PRs generated by AI coding assistants are strealing merit, and even posters talking about the need to tell apart AI PRs from AI-free PRs with proof of work schemes just to feed the merit angle.
I explained GitHub to a friend as "like DeviantArt, but for code".
And it is true, but that just means the bar for expectations has been raised thanks to early open source success. Tarball drops don't cut it anymore (unless you're Alphabet, Inc.) and you actually have to foster a community around what you release in order to attract bug reports and contributions. And you need a code of conduct, and a code of conduct enforcement committee, and a steering committee or board of directors, and...
You just scratch your own itch and you're done. There is no need to "attract contribution", usually, as usually, your project will do exactly what you need.
I think opensource has decided to become big corporate cheerleader for a while and not actually done things strictly in the interest of users or developers, but providing the ideological justification as to why big companies can use their code and not give a dime back. I think aside from free software there could have been a movement where contributing to a free-software project would have allowed contributors to get payed and this would have to preclude companies simply using code as they can in say the MIT license and not give back.
Instead opensource is full of cynical projects and real important projects that do amazing work with maintainers who shoulder a great burden having to listen to people bitch about them for features and fixes while giving nothing and dealing with bad PR's (or even just having to evaluate good and mediocre PR's). Often the only pay is that you can go to HR and shit corp with your CV and say "Here is what I have done, can I now be treated like a bitch for more money".
I think opensource has allowed a lot businesses to treat developers with contempt and this is fully manifest with the way they conceive of AI agent's and Open source has probably become a vector for demoralization and devaluation more then anything.
When technically correct PRs (actual fixes, harmless) get rejected, maybe it's a signal that the codebases are too big?
My big hope for LLM coding is that it will finally force us to focus on minimizing the amount of code that is out there instead of ever producing more.
I also maintain a popular open source repo and have seen similar. If I receive a low effort single-pass obviously-Clauded PR (many people (agents) don’t even bother using the AGENTS.md file I provide right in the repo?!) with no associated issue I also have no qualms closing. I do generally leave a short note about why I closed (volume of these is around 5/week) showing how far off they (Claude) was and encourage opening an issue where we can start a discussion which rarely happens for the drive-by folks. So I totally feel this but I find myself more disheartened by the practice than angry. It feels like slowly watching open-source go the way of email. Open and free until no-cost spam ruined the inbox for everyone...
> If I receive a low effort single-pass obviously-Clauded PR (many people (agents) don’t even bother using the AGENTS.md file I provide right in the repo?!)
Claude doesn't read AGENTS.md automatically, it only looks for CLAUDE.md.
Both are fine, but I also prefer 'furnish', as in put in the main parts of your CV. In a way, 'burnish' is more like 'polish', suggesting an extra layer on top, rather than the fundamental parts.
Just hold off on merging them. When enough typo PRs accumulate, squash all their commits into one, with yourself as Author. Give credit to the contributors in the commit message.
Exactly. Why are people looking for problems here?? There isn’t any. There wasn’t 100s of PRs so why are you so worried about that?? Stop assuming there will be unless you stop even the useful ones.
This makes sense. Apart from the original thing, I no longer upstream anything. It just takes comparatively more effort than maintaining a private fork with my own idiosyncratic fix. This must mean that even small fixes that non-contributors would make in the past are just happening off the books, so to speak. e.g. I upstreamed a tiny fix to Thrift codegen because some function could be much faster. These days I wouldn't bother. I'd just fork and leave upstream to be upstream.
I suspect many people are like me since it feels like a very normie position to take. That means that contributors are even more likely to be useful because both the drive-by genuine contributors have chosen otherwise and these contributors have increased.
It must be quite painful to be an open source maintainer right now. One wonders what to make of such projects in a future where a feature-list is a sufficient prompt.
Discussing the issues, achieving consensus, making sure the fix fits the existing patterns and actually warrants the change, etc. These are and have always been a challenging part of contributing to open source. Typing code in your private fork that satisfies your “idiosyncratic fix” has always existed and been always the easier aspect of such changes. Chances are many of these changes would never be accepted into the repository anyway in the past.
Certainly, but there was motivation to do that in the past. My patch would probably line up with the rest of the code in the direction that they desired and the cost of fork maintenance would have pushed me to upstream in a way acceptable to maintainers.
I ran through exactly this thought process & LLM based resolution just 8 hours ago with a segfault compiling a project which likely doesn't care about the given platform combo much.
Classic problem. I recall spending a long time trying to get Freespace 2 SCP working on the Via Unichrome onboard GPU I had decades ago. Who could possibly have cared for that? The intersection of the two sets was probably precisely one person.
By the way, interesting person to have 512 IPv4s (and two /44s? Aren't those enormous?). Root-level http intentionally user/pass gated on your site? I was curious to see.
Having seen this happening in the past, pre-AI, the answer is usually that they don't. They just stay on that version forever. Or it's maintained until the original author moves on, after which it's someone else's problem to figure out how to move back to the upstream, which has now had years of changes.
Do recruiters actually care about your open source contributions? Especially now when only LLMs read CVs and match them against strict criteria, I doubt there are many companies that actually care about your work outside of work, in fact they might care in opposite direction - thinking that you'll be distracted from work
Just speculating (I use zero automation when recruiting), but automated candidate ranking systems could well still look for signals that were valuable years ago, but no longer are. It's not a given that people using this stuff even understand the criteria under the hood.
Same story as SEO, I presume. Google surely understood their ranking criteria deeply, yet couldn't exactly win that fight.
Google could have easily won that fight by blacklisting scrapping spam sites and spam portals. They just didn't.
There weren't that many with high ranking. I could filter out most of them in like a week of working normally and just right clicking them with the blacklist extension in Firefox.
Nowadays it is probably harder though when Google have broken the internet culture.
If you blacklist them just for yourself, the sites don't care, no economic impact. If Google does the same, they'll work around. There's lots of spam-only sites, but they also often cleverly avoid detecting by mixing in content ranked as valuable/legit. SEO also certainly "attacked" otherwise legitimate sites like Reddit, although what GEO is doing there is far worse I imagine.
I was thinking the same. The amount of effort people went through just for a t-shirt. Now you can try to boost your career, far more valuable, and even with less effort. Of course it will happen, unfortunately.
Seems like a contributor reputation score (shared across all projects) that one can build up over time would be a clean solution to the low quality pull requests.
> raised three separate PRs to correct spelling and grammar mistakes in comments.
> The changes were harmless and correct,
> I closed all three PRs without comment.
What is this article? I get real ‘AI slop PRs’ are nightmares - but I take that to mean the wave of AI generated PRs that read convincingly but are in the end bullshit.
But your example is NOT that. The unreasonable counter to your point: don’t make spelling mistakes in the first place? Or switch off PRs if you don’t want things corrected?
There is a statistically significant correlation between comments on HN not getting the point and a GitHub user existing under the same moniker with a default pfp and no content.
Vibe slop or put on your C.V "contributor to <wellknownproject>"? In terms of social currency B wins easily, unless you can guerilla market as well as Openclaw.
that won't work because all those millions of 'projects' are unmaintainable vaporware which is exactly why people try to piggyback on the reputation of software that is actually being used and developed by people
GitHub flavored FOSS I believe works best for "corporate FOSS" projects and terrible for "FOSS as how it was conceived decades ago".
Or rather I believe that it is engineered exactly for the former.
What it does well is coordination between corps, working in public (while on a corp payroll) and extracting some drive-by-PR value out of random third-parties.
The closer your project is to that shape, the better it works for you. The further it is away, the more pain you will feel.
This has nothing to do with AI slop.
AI slop just turned up a few knobs that were already there.
I very much can relate, as I've also been receiving many of these drive-by PRs lately.
One big problem I have with them is that they take away time from project maintainers for reviewing and helping to get the PRs into shape, which then can't be spent on other, more important things. I feel like the "good first issue" GitHub label is specifically attracting these kinds of contributions.
It's not a black-or-white thing though, and you need to tell apart folks who produce slop PRs against any arbitrary repo, from folks using AI to contribute in a sensible way. We've tried to codify some rules in our contribution guide [1]:
- PRs from apparent bot accounts are closed
- PRs from users who file large numbers against random repos are closed
- You're welcome to use AI, but you need to stand behind your PR and be able to explain it
I'm sure we'll adjust those rules over time, but since we have instantiated them, it definitely has become easier to deal with AI PRs and handle them in an a relatively objective way. It absolutely means that sometimes a PR will be closed which could have been an improvement, but I think this is the right thing to do given the circumstances.
I help maintain a big project as well and I can totally relates what the author is describing.
The amount of security advisories and PRs opened is getting out of hands.
This is no longer just spell check PRs that you can merge in less than 1 minute of review. Often that's going to be supposed perf improvement, attempts at fixing things that are not broken, or submitting completely new features.
These relatively low effort PRs takes a lot of effort to review or even just to filter out.
AI code review is nice to have an additional pair of eyes but it doesn't substitute to the maintainer eyes.
A big part of a review is deciding if you want that change merged in. Not because of the immediate code but what it means to the project to bring this in.
AI code review, to my eyes, just reduce the number of bugs, they do not shorten the review time. Otherwise, it means you are compromising on the direction of the project.
> The changes were harmless and correct, but that did not make me feel better about accepting or merging them.
So do you have the project’s best interest at heart or not? If you’re more concerned about the intent of a valid contribution than the content, why don’t you ask yourself where your intent is? You rejected a valid contribution based on unverified vibes about the person’s intent, instead of assuming they were just being helpful.
Why not assume someone does have the best interest of their own project at heart? How do you get to question that out of the gate, while pretending automated grammar and spelling fixes to rack up PR counts are benevolent unless proven otherwise? Simply assume the person who made the thing you didn't make, who was the only person (or persons) on the planet to come up with that exact thing, knows what's best for that person (or persons). Just like you would handle your own stuff.
I think it’s implied that accepting these kinds of PRs will take time away from working on actual improvements to the codebase. So, on the long term it’s better to be strict with these kinds of silly contributions, and focus on changes that matter more (e.g. features or fixes).
Ironically this seems like a perfect use for AI from the maintaner side. "Find low-effort/AI-like PR's without associated issues from new contributors. Discuss with the contributors in the PR about how contributions should be made, then reject their PR's. If any bad behavior is detected in this interaction, block the accounts for 24 hours and add them to a list of accounts to review for permanent banning from the project".
Automated PR's can have automated responses. You make a human effort, you get a human response.
We do similar in Homebrew by autoclosing opens without the issue template (which implies using the API which often now implies AI).
The weird flip side is that the average good AI contribution is better than the average “I used no AI” contribution. Perhaps it’s because Homebrew has so many declarative guardrails and is so easy for agents to test.
I think the idea of leveraging AI to find useless PRs is right.
I didn’t like the second part though where it starts banning people for “bad behavior”.
I get that AI resistance will be a thing in the next few years, but realistically engineers who plan on being around for the next 20-30 years just need to embrace it instead of being sour about it.
Because of machines, things have arguably improved for humanity as a whole. People who seek more elevated/finer products can still turn to handcrafted products.
Mass software will be produced by AI, and it will scale just fine.
> Because of machines, things have arguably improved for humanity as a whole.
That's a very narrow view of the situation. I agree that this side of the coin exists, but there's also the other side where drinkable tap water is becoming a luxury in much of the global North, giant tornadoes and fires are sweeping the Earth, and temperatures are growing out of control.
So, to make lives more comfortable with machines for some millions of privileged people, they triggered the 6th mass extinction. That's the other side of the coin.
I just think when you close PR's outright, you tend to get angry responses. And as a maintainer, you don't want to deal with angry responses. People who post bot PR's but can't accept having their PR's closed by a bot, shouldn't be visible to the maintainers.
Which part of the second part do you dislike more?
That AI will be given the power to ban people?
Or that AI will be the ones getting banned?
If it's the former, expect an invite from the AI resistance. :)
If it's the latter, well, that can be fixed by spending more tokens and writing better PRs so that the AI moderator doesn't think they're "low-effort, AI-like". I've seen my share of low-effort PRs from humans, too. Low-effort is low-effort, whether it's from humans or AI.
> I didn’t like the second part though where it starts banning people for “bad behavior”.
I think you failed to notice the ban would be a reaction to recurrent bad behavior, such as ignoring maintainer requests.
What's your solution to handle accounts which spam projects with nonsense PRs?
> get that AI resistance will be a thing in the next few years (...)
This is a very lazy opinion to express. If you pay attention to the post you're replying to, you will notice that the problem isn't AI. The problem is spamming projects with PRs that are meaningless while completely ignoring maintainers.
You'd have the same problem if some guy decided to spam the project with a constant barrage of small low-effort nonfunctional PRs.
This is perfect.
Since AI makes it easier for incompetent people to look smart, AI can help us expose those people for what they are.
As a contributor who had to deal with 2 automated reviews in the past month, it's a bad experience. I really wanted to upstream my changes, but these bots made me question if i should at all.
Bots don't understand how their words can hurt people's feelings, and can even less understand that talking to a LLM is even more sad/infuriating for actual humans. I can handle CI/tests just fine, but LLMs playing on my human emotions will have me swearing in no time.
I contribute to free software projects because i want user control over the machine (that was the idea, long before LLMs). Slopware might technically be FLOSS, but it's certainly not in the spirit of FLOSS.
While exploring an open-source project recently, I noticed a bug in the initial setup. I asked Claude to verify it, and the AI confirmed the issue. When Claude asked if I wanted to file a bug report, I agreed, assuming it would simply generate a formatted text block for me to copy and paste. However, I soon realized it had actually submitted the issue directly to GitHub. It turned out I had the GitHub CLI installed and was logged in with my work account.
Out of curiosity, I then asked Claude to draft an enhancement request for a new feature. It created a highly detailed issue and went on to generate the corresponding code for the enhancement.
I haven't submitted a pull request because the code is written in a language I don't know. Part of me considered submitting it anyway since it works and could be useful, but after reading this post, I think I'll just keep it under wraps.
> it had actually submitted the issue directly to GitHub
Did you fix that? Apologize to the people running the repo and all?
IMHO, the days when open source contributions are often positive signal for hiring are long gone.
Actually, if I see someone doing open source like it's a performative career checkbox, that will not be positive signal, and could easily be negative. I understand that people will do what they need to do to get a job, but that pragmatic career checkboxing itself isn't positive. I will have to look for positive signal elsewhere for that person.
The problem is that our industry has gotten very bad at hiring, and now we have everyone playing all sorts of games with it -- rehearsing Leetcode interviews, bad faith open source contributions, feigning enthusiasm, spamming AI-tweaked resumes, outright cheating on interviews -- rather than focusing on doing good work, and being part of a team.
If you're involved in hiring, and you care about effectiveness and culture, consider pushing back against the prevailing dysfunctional big-corporate-cattle-herding practices. Especially if your company has no excuse to be big-corporate dysfunctional, and can't afford to be.
Doing good work leads you nowhere. You need to be in the game of power. Be visible, know every 3 levels of managers by name. Otherwise you’re only number in payroll’s sheet. Modern tech companies are not different from world war 2 era factories imho. Account xyz@bigcompany pushing lines of code to endless repo. No face, no name. Boring. Boring reasonable and reliable. That’s not what hiring people are looking for. You must be a rockstar, conference speaker and leader of few open source projects now. Skills, experience and good work ethic is not enough.
Open source contributions as a signal of developer quality is a thing of the past.
Open source contribution on github specifically is no longer about building and maintaining something, its a parasocial signal that has more in common with linkedin hustle bloggers.
So the fixes are still fixes, but we (I am also a OSS maintainer) are unwilling to accept them as they boost the contributor’s status where we think the merit is very or extremely limited.
Why not have these PRs counted differently (by the platform), and/or colored differently in the timeline(s) thus made less visible or more clear?
Change is bad unless it's great.
Unless the change is an obvious improvement, it has to be worth the time for the maintainers to spend attention reviewing it (and supporting the code forever, and all the rest).
Even if these particular changes are "harmless" and easy to review, accepting them sets a precedent that encourages an unsustainable flood of AI-generated changes that will overwhelm the project.
That’s a totally unnecessary high bar you’re setting. Most change is mediocre (a tautology, I realize). I really don’t believe that you can think that fixing a bunch of spelling mistakes is not a small improvement.
About setting a precedent: if you think like that you probably never accept any contributions at all, by humans or otherwise. Contributions by strangers have always been very minor things in general, things that the author cared enough about to make a pull request. In this case I don’t see the difference except for the fact that ai was used. If you just don’t accept ai at all , fine. But just be clear about it. In this case they are calling spelling correction slop. That’s not what slop means. Pretending human contributions were always “great” and nothing else would do is kind of ridiculous and shows a lack of experience in open source.
It's not an unnecessarily high bar.
Change is bad. It takes engineers time to understand and approve. It confuses users. Changes should be made for good reasons, not frivolously.
Just because writing code is easier does not mean you should write more of it. It is more important than ever to carefully consider what you work on and what work you accept.
The user says "this is just spelling corrections" but a maintainer - a volunteer - needs to read all the files and make sure it's harmless and doesn't introduce security vulnerabilities, change meanings, and so on. That's not free.
These fixes, regularly destroy the architecture, accrue bloat for little gain, refuse to rewrite while demanding to rewrite- and many other such funny noises. Most code contribution by LLMs is garbage if you long-term care about the project. Look at closed source projects that ingest all this madness - windows with its seconds to open the explorer and other catastrophes.
Why not let them have the status boost? This isn't zero sum.
> This isn't zero sum.
True, it is actually negative sum, because fake merit erodes trust in real merit, as it makes it a lot harder to spot the latter.
I think Goodhart's law applies really well here. When people make commits for the sake of having made a commit to some famous repo, the "merit" that a commit measures is already useless. Remember Hacktoberfest 2020, before LLMs were popular? https://news.ycombinator.com/item?id=24643894
What merit was there to have anyway. Would the author accept the same changes from a human?? I am inclined to believe they would, I certainly would. Would you have attributed merit to someone just fixing typos? I mean it’s quite helpful. But I wouldn’t think a commit fixing typos ever attributed merit to a developer. So it’s hard to understand the problem with AI doing this. If it was a bot like dependabot but for fixing grammar errors would that be a problem? I assume no, hence the problem is entirely that the human author of the PR is being attributed nonexistent merit by people who should know better.
Well, as I mentioned there was already in 2020 the problem of (human-generated) slop commits causing unnecessary work for maintainers.
The commit in itself might add some minuscule value to the project, but when you take into account the time it takes to check the commit and approve it, you're probably back at negative value again.
Plus, there's the problem of people doing this in an adversarial way. How can you tell whether a random commit touching 100 files is the work of an enthusiastic beginner, or a malicious hacker that's hidden a subtle attack somewhere? Well, you need to spend your time carefully checking it.
So it's way more pragmatic just to set up some automated filter to put all of this stuff in the bin.
Oh and by the way:
> I wouldn’t think a commit fixing typos ever attributed merit to a developer.
The first sentences of the article in question:
> Successful contributions to open source projects are a kind of currency. GitHub in particular encourages this in a number of ways: [...] Potential hiring managers often take note of this. Recruiters often find and screen candidates this way.
Because letting sloperators near your codebase, even for a very minor obvious and correct change, may have long-term negative consequences for your project. It sends a signal to other sloperators to keep doing what they do, when we really need to send them a signal to get a life, learn a craft, and let Claude burn in hell.
Because every merge request and every merge comes with cost and (potential) more cost in the future.
Linux lately received a lot of fixes for drivers nobody has cared about for ages. They decided to remove them. Maybe they should have done this earlier, I don't know, but that happened because every code change comes with cost down the line. For real contributers, this cost is much more limited and they have a higher probability of burden the cost down the line.
Pure AI MRs are just rude, because it's just "here take this, don't really care what this is, but AI said it being good, I won't be around, so you will have to deal with this code after the merge and you better don't overlook anything, because I personally don't really know what I am doing, I only know how to have AI making something that looks convincing. Good luck fixing the bugs this change introduces".
Boost for what? That they can type a prompt?
How do you distinguish those who know what they are doing from those who only can write a prompt and copy the result?
Making the haystack bigger is a bad idea if you search the needle
The whole idea of "counting PRs" as a vanity metric is flawed because vanity metrics are flawed.
I don't think that there is a technical solution to be found here, as the problem is anything but technical.
__
A hack/trap:
Comment "Ah yes thanks a lot for the hint :)", then make the changes yourself.
Then see how the person reacts to that.
Hack the grifters. Hack the planet.
My solution is to ask them for proof of work. Many of these disappear after opening a PR that they never tested. How can you claim to have fixed a visual issue without a screenshot? Insta-close.
I have noticed a lot of my Github followers who are either newer students or just new to Github have an immaculate green board of commits on their profile page, but it's so over the top I only like the ones that have creative illustrations or just very limited edits. Some seem so automated that it's hard to know how much they actually work on it. Others even are programmed to follow accounts then drop off once the person doesn't reciprocate, which is technically better than them staying but also systematically checking what sticks (also Github should limit some of these automated follows).
Or just pull the pr into your local git, git amend --reset-author, and then push it to master yourself.
Merging commits/PRs without reading them is how you get a 2024 xz situation.
Not just that how the maintainer was treated by the attacker which tells you something about the way people feel entitled to maintainers time and efforts for nothing even by people not looking to exploit the project. And the abuse after he received despite it being an easy mistake that the culture encourages people to make.
> And the abuse after he received despite it being an easy mistake that the culture encourages people to make.
I believe that that was in part simply people defending their reality.
Because if it's not the maintainers fault, then why did it fail? What are the implications of that? What does it mean for me personally? Do I have to do something differently? Do I have to change my assumptions?
People do not like these feelings. They want to simply push them away. The easiest way of doing so is blaming that guy and being done with the negative emotion.
Most people do suck at this whole "being human" business.
I never suggested that you shouldn’t check the commit before merging!
It still haunts me, that dot they slipped in, in one of the commits, every time I merge a PR
I think what you're saying is fair, tbh. Imagine you put in months or years into a project to make it a quality piece of work. It develops into something notable and you took all the risk. Then someone comes along to fix a spelling error with a pull request so their name effectively appears on the repo as a "contributor." And you just know right after its going on their resume as "contributed to [...]" or maybe if they're bold "software engineer working on [...]" which implies substantial investment. Then you're effectively sharing credit for YOUR work with someone who did nothing. That is rage inducing. ((Of course: it probably is just juniors trying their best in this horrible industry.))
On the other hand: lets be careful not to dismiss valid but inexperienced attempts to contribute. Having someone want to genuinely contribute to your software is incredibly generous. If someone seems like they're trying its better to give advice than act like a snob because its not good enough. Often pull requests only need small fixes to get in, anyway.
Hi Neil, fun to see you on HN. I agree with your points and I you summarized it very well as "Ultimately, open source is built on trust".
AI is destroying trust in open source and many other areas and I think this will discourage teams from publishing their source code in the future.
On the other hand, personal connections are becoming even more important, which is unfair to the younger generation and people who don't live near tech hubs.
Today's conception of open source is different from the original anyway. It used to be "we made this thing, here's the code" but now it's turned into some kind of weird parasocial thing more akin to Facebook.
> Today's conception of open source is different from the original anyway. It used to be "we made this thing, here's the code" but now it's turned into some kind of weird parasocial thing more akin to Facebook.
I didn't agree with your comment when I glanced through it, but then further ahead in the discussion I stumbled upon a whole thread of people commenting that PRs generated by AI coding assistants are strealing merit, and even posters talking about the need to tell apart AI PRs from AI-free PRs with proof of work schemes just to feed the merit angle.
Crazy times we live in.
I explained GitHub to a friend as "like DeviantArt, but for code".
And it is true, but that just means the bar for expectations has been raised thanks to early open source success. Tarball drops don't cut it anymore (unless you're Alphabet, Inc.) and you actually have to foster a community around what you release in order to attract bug reports and contributions. And you need a code of conduct, and a code of conduct enforcement committee, and a steering committee or board of directors, and...
You don't need any of that, usually.
You just scratch your own itch and you're done. There is no need to "attract contribution", usually, as usually, your project will do exactly what you need.
I think opensource has decided to become big corporate cheerleader for a while and not actually done things strictly in the interest of users or developers, but providing the ideological justification as to why big companies can use their code and not give a dime back. I think aside from free software there could have been a movement where contributing to a free-software project would have allowed contributors to get payed and this would have to preclude companies simply using code as they can in say the MIT license and not give back.
Instead opensource is full of cynical projects and real important projects that do amazing work with maintainers who shoulder a great burden having to listen to people bitch about them for features and fixes while giving nothing and dealing with bad PR's (or even just having to evaluate good and mediocre PR's). Often the only pay is that you can go to HR and shit corp with your CV and say "Here is what I have done, can I now be treated like a bitch for more money".
I think opensource has allowed a lot businesses to treat developers with contempt and this is fully manifest with the way they conceive of AI agent's and Open source has probably become a vector for demoralization and devaluation more then anything.
When technically correct PRs (actual fixes, harmless) get rejected, maybe it's a signal that the codebases are too big?
My big hope for LLM coding is that it will finally force us to focus on minimizing the amount of code that is out there instead of ever producing more.
I also maintain a popular open source repo and have seen similar. If I receive a low effort single-pass obviously-Clauded PR (many people (agents) don’t even bother using the AGENTS.md file I provide right in the repo?!) with no associated issue I also have no qualms closing. I do generally leave a short note about why I closed (volume of these is around 5/week) showing how far off they (Claude) was and encourage opening an issue where we can start a discussion which rarely happens for the drive-by folks. So I totally feel this but I find myself more disheartened by the practice than angry. It feels like slowly watching open-source go the way of email. Open and free until no-cost spam ruined the inbox for everyone...
> If I receive a low effort single-pass obviously-Clauded PR (many people (agents) don’t even bother using the AGENTS.md file I provide right in the repo?!)
Claude doesn't read AGENTS.md automatically, it only looks for CLAUDE.md.
True, but I do have a claude.md as well. Point is, best practice agentic development doesn’t seem to be on display.
I only look for humans.md and humans.txt.
Very intelligent
Interesting analogy.
Sounds like an opportunity for GitHub/others to better filter these.
I think the author meant 'burnish' there, which is a clever way of showing they didn't use AI :-)
Furnish makes more sense.
Both are fine, but I also prefer 'furnish', as in put in the main parts of your CV. In a way, 'burnish' is more like 'polish', suggesting an extra layer on top, rather than the fundamental parts.
“The spirits that I summoned, I cannot now dismiss.”
https://en.wikipedia.org/wiki/The_Sorcerer%27s_Apprentice
I've submitted PRs for typos in the (pre-LLM) past, not to boost my GH profile, but because I thought I was being helpful. Guess I was wrong...
Scale matters. If people do it, it's maybe 10 PRs like that per month. With AI there could be hundreds per day.
Just hold off on merging them. When enough typo PRs accumulate, squash all their commits into one, with yourself as Author. Give credit to the contributors in the commit message.
Then that AI is fixing hundreds of typos a day, seems like win.
Exactly. Why are people looking for problems here?? There isn’t any. There wasn’t 100s of PRs so why are you so worried about that?? Stop assuming there will be unless you stop even the useful ones.
This makes sense. Apart from the original thing, I no longer upstream anything. It just takes comparatively more effort than maintaining a private fork with my own idiosyncratic fix. This must mean that even small fixes that non-contributors would make in the past are just happening off the books, so to speak. e.g. I upstreamed a tiny fix to Thrift codegen because some function could be much faster. These days I wouldn't bother. I'd just fork and leave upstream to be upstream.
I suspect many people are like me since it feels like a very normie position to take. That means that contributors are even more likely to be useful because both the drive-by genuine contributors have chosen otherwise and these contributors have increased.
It must be quite painful to be an open source maintainer right now. One wonders what to make of such projects in a future where a feature-list is a sufficient prompt.
Discussing the issues, achieving consensus, making sure the fix fits the existing patterns and actually warrants the change, etc. These are and have always been a challenging part of contributing to open source. Typing code in your private fork that satisfies your “idiosyncratic fix” has always existed and been always the easier aspect of such changes. Chances are many of these changes would never be accepted into the repository anyway in the past.
Certainly, but there was motivation to do that in the past. My patch would probably line up with the rest of the code in the direction that they desired and the cost of fork maintenance would have pushed me to upstream in a way acceptable to maintainers.
I ran through exactly this thought process & LLM based resolution just 8 hours ago with a segfault compiling a project which likely doesn't care about the given platform combo much.
Classic problem. I recall spending a long time trying to get Freespace 2 SCP working on the Via Unichrome onboard GPU I had decades ago. Who could possibly have cared for that? The intersection of the two sets was probably precisely one person.
By the way, interesting person to have 512 IPv4s (and two /44s? Aren't those enormous?). Root-level http intentionally user/pass gated on your site? I was curious to see.
How could it take less effort to maintain a fork? You have to merge the upstream in every week and fight endlessly on conflicts with your patches.
Having seen this happening in the past, pre-AI, the answer is usually that they don't. They just stay on that version forever. Or it's maintained until the original author moves on, after which it's someone else's problem to figure out how to move back to the upstream, which has now had years of changes.
Originally, yes. Nowadays, it's trivial. Agents can handle this flawlessly.
Do recruiters actually care about your open source contributions? Especially now when only LLMs read CVs and match them against strict criteria, I doubt there are many companies that actually care about your work outside of work, in fact they might care in opposite direction - thinking that you'll be distracted from work
Just speculating (I use zero automation when recruiting), but automated candidate ranking systems could well still look for signals that were valuable years ago, but no longer are. It's not a given that people using this stuff even understand the criteria under the hood.
Same story as SEO, I presume. Google surely understood their ranking criteria deeply, yet couldn't exactly win that fight.
Google could have easily won that fight by blacklisting scrapping spam sites and spam portals. They just didn't.
There weren't that many with high ranking. I could filter out most of them in like a week of working normally and just right clicking them with the blacklist extension in Firefox.
Nowadays it is probably harder though when Google have broken the internet culture.
If you blacklist them just for yourself, the sites don't care, no economic impact. If Google does the same, they'll work around. There's lots of spam-only sites, but they also often cleverly avoid detecting by mixing in content ranked as valuable/legit. SEO also certainly "attacked" otherwise legitimate sites like Reddit, although what GEO is doing there is far worse I imagine.
In my experience no, it's usually just something you mention during the tech rounds in passing.
I’ve heard about similar issues with Hacktoberfest’s T-shirts back in 2020, but it was not as automated back then: https://news.ycombinator.com/item?id=24643894
I was thinking the same. The amount of effort people went through just for a t-shirt. Now you can try to boost your career, far more valuable, and even with less effort. Of course it will happen, unfortunately.
Seems like a contributor reputation score (shared across all projects) that one can build up over time would be a clean solution to the low quality pull requests.
> raised three separate PRs to correct spelling and grammar mistakes in comments.
> The changes were harmless and correct,
> I closed all three PRs without comment.
What is this article? I get real ‘AI slop PRs’ are nightmares - but I take that to mean the wave of AI generated PRs that read convincingly but are in the end bullshit.
But your example is NOT that. The unreasonable counter to your point: don’t make spelling mistakes in the first place? Or switch off PRs if you don’t want things corrected?
There is a statistically significant correlation between comments on HN not getting the point and a GitHub user existing under the same moniker with a default pfp and no content.
This is weird though.
Obviously the best way to use AI to furnish your CV is to build your own project using AI.
Vibe slop or put on your C.V "contributor to <wellknownproject>"? In terms of social currency B wins easily, unless you can guerilla market as well as Openclaw.
that won't work because all those millions of 'projects' are unmaintainable vaporware which is exactly why people try to piggyback on the reputation of software that is actually being used and developed by people
GitHub flavored FOSS I believe works best for "corporate FOSS" projects and terrible for "FOSS as how it was conceived decades ago". Or rather I believe that it is engineered exactly for the former.
What it does well is coordination between corps, working in public (while on a corp payroll) and extracting some drive-by-PR value out of random third-parties.
The closer your project is to that shape, the better it works for you. The further it is away, the more pain you will feel.
This has nothing to do with AI slop. AI slop just turned up a few knobs that were already there.
These maintainer who’ve held the sacred fire for years…
These idea guys who longed for the fire for years- and still don't have it.. full of envy..
I very much can relate, as I've also been receiving many of these drive-by PRs lately.
One big problem I have with them is that they take away time from project maintainers for reviewing and helping to get the PRs into shape, which then can't be spent on other, more important things. I feel like the "good first issue" GitHub label is specifically attracting these kinds of contributions.
It's not a black-or-white thing though, and you need to tell apart folks who produce slop PRs against any arbitrary repo, from folks using AI to contribute in a sensible way. We've tried to codify some rules in our contribution guide [1]:
- PRs from apparent bot accounts are closed - PRs from users who file large numbers against random repos are closed - You're welcome to use AI, but you need to stand behind your PR and be able to explain it
I'm sure we'll adjust those rules over time, but since we have instantiated them, it definitely has become easier to deal with AI PRs and handle them in an a relatively objective way. It absolutely means that sometimes a PR will be closed which could have been an improvement, but I think this is the right thing to do given the circumstances.
[1] https://github.com/hardwood-hq/hardwood/blob/main/CONTRIBUTI...
This is nothing new, people have been making spellcheck commits all over github for years... and that was before ai slop even existed.
I help maintain a big project as well and I can totally relates what the author is describing.
The amount of security advisories and PRs opened is getting out of hands. This is no longer just spell check PRs that you can merge in less than 1 minute of review. Often that's going to be supposed perf improvement, attempts at fixing things that are not broken, or submitting completely new features.
These relatively low effort PRs takes a lot of effort to review or even just to filter out.
Create AI to review the pull requests and auto ban people for low SNR.
AI code review is nice to have an additional pair of eyes but it doesn't substitute to the maintainer eyes. A big part of a review is deciding if you want that change merged in. Not because of the immediate code but what it means to the project to bring this in.
AI code review, to my eyes, just reduce the number of bugs, they do not shorten the review time. Otherwise, it means you are compromising on the direction of the project.
I meant AI to judge whether a PR is trash or not for ban purposes. Not AI to do the thorough code review itself.
> The changes were harmless and correct, but that did not make me feel better about accepting or merging them.
So do you have the project’s best interest at heart or not? If you’re more concerned about the intent of a valid contribution than the content, why don’t you ask yourself where your intent is? You rejected a valid contribution based on unverified vibes about the person’s intent, instead of assuming they were just being helpful.
Why not assume someone does have the best interest of their own project at heart? How do you get to question that out of the gate, while pretending automated grammar and spelling fixes to rack up PR counts are benevolent unless proven otherwise? Simply assume the person who made the thing you didn't make, who was the only person (or persons) on the planet to come up with that exact thing, knows what's best for that person (or persons). Just like you would handle your own stuff.
The changes were grammar and spelling fixes in comments. It is not sufficient for someone to intend to be helpful.
I think it’s implied that accepting these kinds of PRs will take time away from working on actual improvements to the codebase. So, on the long term it’s better to be strict with these kinds of silly contributions, and focus on changes that matter more (e.g. features or fixes).
If people can spam low-effort PRs they will. Unverified vibes is you closing your eyes to a very real issue.