I know, I know: It's complicated. But have you heard of AI agents?
But I just onboarded 4 interns on a project where all they had to do was
1. Install the Nix package manager
2. Install direnv, enter the project repo, and `direnv allow`
3. Toolchain, git hooks, MCP servers, in-repo issue tracker, everything is available
Our project manager requested information that was available in the issue tracker. I told her, she could get all her answers by asking our agent, and it'd automatically reference the issue tracker. I figured I'd just need to show her how to install the Nix package manager. But no, she already had it because another project by another team depended on it.
Putting wrong information is README is so outdated when you have programmatic setup of your entire toolchain.
The link says "My opinion is that FAQs pose a problem only when there’s no strategy around their usage." Calling them "problematic" based on that seems like an exaggeration
> My jokes aren't funny and are actively confusing.
I used to write technical documents in prose style, sometimes with meandering stories. I guess I picked it up from my early blogging days. I realized I hated reading some of them back. So I tried to keep it cut and dried. I do sometimes sprinkle a bit of colorful wording just to add a bit of humanity but only if it doesn't get in the way of the main message.
Being short and concise is usually the better way.
I tend to be too verbose in writing as I want to explain the context in more detail, but short, accurate, concise is best. Many developers fail at that too, though. Many projects do not have working examples. That annoys me the most. It sends a message of "I don't care about new users learning how to use my project".
Most documentation reads like you should already know what you’re doing, which makes sense because it was written by someone who already knows how to do the process.
I think good technical writing requires the same skills as good product ownership, that is empathy for the user and their perspective. Often technical writing is an after thought and not someone’s whole role and it really shows.
This sounds exactly like UX usability testing. Sit down with someone and watch them work through whatever process you’ve created.
It’s normal to compensate them for their time.
Normally though you don’t modify it after each participant. But for something very niche like following a README (as opposed to an e-commerce flow targeted to the general population) it might be fine, if less rigorous.
There's a zeroth step missing from both the Quickstart and Full Set Up: install php-fpm, configure it, and configure your http server to serve using it as a backend.
It seems to be taken for granted — that's something you'd already have in place if you're already serving applications in PHP, but would have to figure out on your own if you haven't served anything in PHP so far in your life.
It's a tricky problem. How far back in the stack do you go? The README assumes that you know how to use git to check out the files - or that you can easily save them from the repository. Should it include that as a step in the tutorial?
As I say in the linked article, it depends on what sort of user you have. For a "getting started with Raspberry Pi" document, you might well want to include how to insert an SD card etc.
I'll have a think about the best way to help people figure out if they're running PHP. Thanks for the feedback!
Feedback and issues you need to troubleshoot with your projects is a good indicator of your audience level, and from my experience it’s helpful to understand that documentation is always under development just like the code
from experience in ENT support where i was sending instructions & quick fix scripts to technically capable persons, you will learn very fast when you’ve missed the mark with your documentation / instructions. tons of times i had ready made solutions that i thought “just copy and paste and go what could possibly go wrong?” and was caught off guard how often a little too much knowledge lends to confusion. i am not blaming the users here it is my fault that i didn’t explain things like “no don’t change this date in the fix that is a special date when the issue could have earliest occurred and it’s there to avoid grabbing more than we need to parse”, but i didn’t tell that so of course people changed it to all sorts of dates thinking they had to
such feedback and issues also got me way better about writing code that avoided chances for such mistakes as i didn’t want users to have to read a novel to understand what to do; it’s a fine balance between what to solve with documentation and what to solve with code
Installation instructions usually (should) have a “prerequisites” section. You don’t have to explain how to install the prerequisites, but they should be listed.
My rule of thumb for my READMEs: there should be a list of commands, that when executed in order and in a clean machine, result in the software doing something useful. Yes, this includes `git clone`.
If there's something the user might already have, like the webserver, I add a comment "skip this if you already have a web server". If there are any shortcuts that make it not production-ready, it's time to break out the ALL CAPS.
Limiting the operations to simple commands also helps me keep honest about the instructions (no hidden assumptions), and forces the software to be minimally testable.
Generally it seems like good READMEs assume you have a compatible OS ready to go, but will give you a summary of all commands to get a working setup from there.
Even as someone working with PHP I would prefer if the project provides we with some guidelines for setup. Especially if it requires some specific version or extension. Ideally the whole dev environment should be containerized. Then you would again require people to understand and use that layer, but depending on the projects complexity definitely something to consider to make it easier to work with
Yes. The amount of projects that don't just run is outstanding. Luckily docker container projects are inherently better at this is in terms of dependencies, but there are still often weird assumptions or medical incantations to get them during.
"They were the ones who caught the mistakes that no spell chequer could."
Nice, good article lol. I appreciate a joke or two in a readme but totally understand the annoyance, I think forgetting to actually say what the software does is super common as well though. Often trying to figure out if something found on github will actually solve a problem only to be met with a list of install commands.
The 'look awesome project with funky name, here's how to install off you go' without a word to what one earth it actually does is amazingly common. Plenty of things posted here I bounce away from after hitting the linked page, finding something someone is obviously very proud of, but not having a clue what it's actually about :p
> In total, I paid out around €150 to have a bunch of people criticise me to my face. Hey, cheaper than therapy, right?
My first thought was “no, not really, and of course it’s far from the same thing, I wonder where the author is that one therapy session would cost this much and…”, at which point I stopped myself and remembered another line earlier in the article, from the things the author learned from the experiment:
> My jokes aren't funny and are actively confusing.
Somewhat related question: what is it about READMEs that AI agents love to dump the most useless, hard to contextualise & comprehend, irrelevant rubbish into them that makes understanding a project and onboarding so hard?
It feels like like the AI agents can't help themselves sometimes, and the judgement exercised around what's included and omitted is baffling.
But maybe READMEs have always been this bad, and AI agents have raised the baseline?
Indeed, these make me back out of a project immediately if I don’t have a strong need to use it, and it takes extra cognitive effort to ignore the emojis and focus on the actually meaningful text.
This one wasn't as bad as the majority that have the emoji in each header too. At least these emoji did seem to properly represent the item they're next to, not an immediate rocket ship emoji in the header following by irrelevant ones of various sizes littered throughout the text.
To me it’s similar to what you wrote about the jokes you removed: the emojis are distracting and not actually fun. This use of emojis comes across as a tired meme. It’s better if the project makes people smile due to its intrinsic qualities like working well and doing what they want.
Writing good documentation is difficult. From those who say
"the source code explains everything", I think 80% are too
lazy to write documentation in the first place.
Having said that, I found consistently that when a project
has working examples, ideally documented a bit, aka explained,
they tend to work much better than those projects that have
no examples. Working examples often also help get into a project
quickly and check out how it works. It helps to learn too.
READMEs are not useless, of course, but the quality varies a lot.
I also know of folks who use AI slop spam to improve it, but while
it may improve a little bit, it generates a lot of horribly to read
text that makes no sense. I am noticing this with the ruby core dev
team - they (almost) all suddenly have perfect language skills but it
is more like an advanced babelfish translator. What they piece together
here makes no sense. Claude in particular is now famous for this slop
content. And I don't understand what it is used: real people read any
of this AI slop? Because I just skip it or filter it away these days.
>From those who say "the source code explains everything"
Because theoretically you should be able to describe not only your entire application logic, but upper and lower bounds of inputs as well. Good code would describe this inherently.
Documentation is only useful when a) the application is not source available, so you have no choice, b) you want to save a human developer time for them to understand your code, or c) you want to use documentation as a cache hit for agent use (less token spend)
>But it is really hard to ignore your own biases. Of course you know that certain commands require sudo and obviously when you wrote -foo you meant --foo and everyone knows that you have to reboot afterwards.
For God's sake, this is not "biases," wtf?? That's straight up just not reading/following the document you're supposed to be testing/reviewing. When I test my documents I actually follow them exactly, step by step, and I always catch these sorts of mistakes. Always. I always copy/paste commands because I know that is what the customer will do, so I have to make sure that works flawlessly.
Dude, if you actually follow your own document, you don't have to pay people to do it for you. Also, you can make sure the person you are paying doesn't ignore the document like you apparently do.
I stopped reading, I'm not interested in whatever else this dude has to say. I'm literally flabbergasted.
Maybe it's because my documents have always gone to real paying customers who have to get through this install, and not some hobby project I'm super proud of or whatever and I don't think I'm super clever? Idk.
> I know someone is going to say "why not just ask an LLM to simulate a range of users?" The answer is very simple - I want to speak to real people. People are brilliant! They can make you laugh, you can see their cat when it wanders on to the call, they bring a unique perspective to the problem, and they're really happy when you give them a €25 voucher. Some will gladly do it for free and make you happy!
So what the author actually paid for was to interact with humans and the README checking was secondary - because, really, my own first thought was literally to ask an LLM check and try to follow the instructions in the README and pretty much any decent LLM (including several local ones) would be able to check if they're adequate and even suggest improvements (just don't let them write it for you :-P).
Nix.
I know, I know: It's complicated. But have you heard of AI agents?
But I just onboarded 4 interns on a project where all they had to do was
Our project manager requested information that was available in the issue tracker. I told her, she could get all her answers by asking our agent, and it'd automatically reference the issue tracker. I figured I'd just need to show her how to install the Nix package manager. But no, she already had it because another project by another team depended on it.Putting wrong information is README is so outdated when you have programmatic setup of your entire toolchain.
Besides emojis, I find it too long. READMEs should be succinct and be like a switchboard to other docs (much like LLMS.txt tries to be for agents).
Also, it features an FAQ. FAQs are problematic: https://passo.uno/what-the-faq/
The link says "My opinion is that FAQs pose a problem only when there’s no strategy around their usage." Calling them "problematic" based on that seems like an exaggeration
> My jokes aren't funny and are actively confusing.
I used to write technical documents in prose style, sometimes with meandering stories. I guess I picked it up from my early blogging days. I realized I hated reading some of them back. So I tried to keep it cut and dried. I do sometimes sprinkle a bit of colorful wording just to add a bit of humanity but only if it doesn't get in the way of the main message.
Being short and concise is usually the better way.
I tend to be too verbose in writing as I want to explain the context in more detail, but short, accurate, concise is best. Many developers fail at that too, though. Many projects do not have working examples. That annoys me the most. It sends a message of "I don't care about new users learning how to use my project".
> Being short and concise is usually the better way.
The golang docs are like this. As a novice, you are looking for detailed prose, but as you progress, you come to appreciate the terseness.
Most documentation reads like you should already know what you’re doing, which makes sense because it was written by someone who already knows how to do the process.
I think good technical writing requires the same skills as good product ownership, that is empathy for the user and their perspective. Often technical writing is an after thought and not someone’s whole role and it really shows.
Good article
This used to be called "I hired QA people to identify gaps in my project", but hey now it has become paying people to follow readme
This sounds exactly like UX usability testing. Sit down with someone and watch them work through whatever process you’ve created.
It’s normal to compensate them for their time.
Normally though you don’t modify it after each participant. But for something very niche like following a README (as opposed to an e-commerce flow targeted to the general population) it might be fine, if less rigorous.
There's a zeroth step missing from both the Quickstart and Full Set Up: install php-fpm, configure it, and configure your http server to serve using it as a backend.
It seems to be taken for granted — that's something you'd already have in place if you're already serving applications in PHP, but would have to figure out on your own if you haven't served anything in PHP so far in your life.
It's a tricky problem. How far back in the stack do you go? The README assumes that you know how to use git to check out the files - or that you can easily save them from the repository. Should it include that as a step in the tutorial?
As I say in the linked article, it depends on what sort of user you have. For a "getting started with Raspberry Pi" document, you might well want to include how to insert an SD card etc.
I'll have a think about the best way to help people figure out if they're running PHP. Thanks for the feedback!
Feedback and issues you need to troubleshoot with your projects is a good indicator of your audience level, and from my experience it’s helpful to understand that documentation is always under development just like the code
from experience in ENT support where i was sending instructions & quick fix scripts to technically capable persons, you will learn very fast when you’ve missed the mark with your documentation / instructions. tons of times i had ready made solutions that i thought “just copy and paste and go what could possibly go wrong?” and was caught off guard how often a little too much knowledge lends to confusion. i am not blaming the users here it is my fault that i didn’t explain things like “no don’t change this date in the fix that is a special date when the issue could have earliest occurred and it’s there to avoid grabbing more than we need to parse”, but i didn’t tell that so of course people changed it to all sorts of dates thinking they had to
such feedback and issues also got me way better about writing code that avoided chances for such mistakes as i didn’t want users to have to read a novel to understand what to do; it’s a fine balance between what to solve with documentation and what to solve with code
Installation instructions usually (should) have a “prerequisites” section. You don’t have to explain how to install the prerequisites, but they should be listed.
They are - https://gitlab.com/edent/activity-bot/-/blob/main/README.md?...
No need for Adobe Creative Cloud is a nice touch
From the root comment, it appears to be incomplete.
> How far back in the stack do you go?
My rule of thumb for my READMEs: there should be a list of commands, that when executed in order and in a clean machine, result in the software doing something useful. Yes, this includes `git clone`.
If there's something the user might already have, like the webserver, I add a comment "skip this if you already have a web server". If there are any shortcuts that make it not production-ready, it's time to break out the ALL CAPS.
Limiting the operations to simple commands also helps me keep honest about the instructions (no hidden assumptions), and forces the software to be minimally testable.
Why stop at `git clone`? Why not include `apt install git` and equivalents for all OSs?
Whenever I write documentation, my first step is to explain how silicon can be used as a transistor.
Yeah well I produce home grown silicon in super novae.
'If you wish to make an apple pie from scratch, you must first invent the universe.'
Generally it seems like good READMEs assume you have a compatible OS ready to go, but will give you a summary of all commands to get a working setup from there.
It's really not that tricky at all, every install document I ever wrote has a "prerequisites" section telling you the prerequisites.
It can't hurt to make a sentence or two about assumptions.
Like "This manual assumes that you have a Linux/BSD, a C compiler, GNU Make, and a text editor".
Even as someone working with PHP I would prefer if the project provides we with some guidelines for setup. Especially if it requires some specific version or extension. Ideally the whole dev environment should be containerized. Then you would again require people to understand and use that layer, but depending on the projects complexity definitely something to consider to make it easier to work with
Yes. The amount of projects that don't just run is outstanding. Luckily docker container projects are inherently better at this is in terms of dependencies, but there are still often weird assumptions or medical incantations to get them during.
"They were the ones who caught the mistakes that no spell chequer could."
Nice, good article lol. I appreciate a joke or two in a readme but totally understand the annoyance, I think forgetting to actually say what the software does is super common as well though. Often trying to figure out if something found on github will actually solve a problem only to be met with a list of install commands.
The 'look awesome project with funky name, here's how to install off you go' without a word to what one earth it actually does is amazingly common. Plenty of things posted here I bounce away from after hitting the linked page, finding something someone is obviously very proud of, but not having a clue what it's actually about :p
Excellent article! Thank you for the reminder that I need to be spending more time with users… or hirelings :)
And did it work?
> In total, I paid out around €150 to have a bunch of people criticise me to my face. Hey, cheaper than therapy, right?
My first thought was “no, not really, and of course it’s far from the same thing, I wonder where the author is that one therapy session would cost this much and…”, at which point I stopped myself and remembered another line earlier in the article, from the things the author learned from the experiment:
> My jokes aren't funny and are actively confusing.
Ah, indeed. At this point I genuinely laughed.
Somewhat related question: what is it about READMEs that AI agents love to dump the most useless, hard to contextualise & comprehend, irrelevant rubbish into them that makes understanding a project and onboarding so hard?
It feels like like the AI agents can't help themselves sometimes, and the judgement exercised around what's included and omitted is baffling.
But maybe READMEs have always been this bad, and AI agents have raised the baseline?
I hate READMEs with a gazillion emojis, too much noise.
Cannot agree more. Looks kinda childish too.
It was nice when emoji were used sparingly and with purpose and intention.
As my childhood English teacher said - too much of anything, good for nothing.
Immediate reaction - LLM generated. Hard to imagine a real human picking emojis to match feature description for a long list.
Indeed, these make me back out of a project immediately if I don’t have a strong need to use it, and it takes extra cognitive effort to ignore the emojis and focus on the actually meaningful text.
This one wasn't as bad as the majority that have the emoji in each header too. At least these emoji did seem to properly represent the item they're next to, not an immediate rocket ship emoji in the header following by irrelevant ones of various sizes littered throughout the text.
To me an emoji-filled README is a good sign both the README and the project were generated by an LLM.
That's interesting. In my testing, about half the participants liked them, one didn't, and the rest didn't express a strong preference.
I kept them because they make me smile.
To me it’s similar to what you wrote about the jokes you removed: the emojis are distracting and not actually fun. This use of emojis comes across as a tired meme. It’s better if the project makes people smile due to its intrinsic qualities like working well and doing what they want.
I needed to read your sentence a few times to figure out you don't hate README files. I too dislike emojis overuse in README files.
Writing good documentation is difficult. From those who say "the source code explains everything", I think 80% are too lazy to write documentation in the first place.
Having said that, I found consistently that when a project has working examples, ideally documented a bit, aka explained, they tend to work much better than those projects that have no examples. Working examples often also help get into a project quickly and check out how it works. It helps to learn too.
READMEs are not useless, of course, but the quality varies a lot. I also know of folks who use AI slop spam to improve it, but while it may improve a little bit, it generates a lot of horribly to read text that makes no sense. I am noticing this with the ruby core dev team - they (almost) all suddenly have perfect language skills but it is more like an advanced babelfish translator. What they piece together here makes no sense. Claude in particular is now famous for this slop content. And I don't understand what it is used: real people read any of this AI slop? Because I just skip it or filter it away these days.
>From those who say "the source code explains everything"
Because theoretically you should be able to describe not only your entire application logic, but upper and lower bounds of inputs as well. Good code would describe this inherently.
Documentation is only useful when a) the application is not source available, so you have no choice, b) you want to save a human developer time for them to understand your code, or c) you want to use documentation as a cache hit for agent use (less token spend)
>But it is really hard to ignore your own biases. Of course you know that certain commands require sudo and obviously when you wrote -foo you meant --foo and everyone knows that you have to reboot afterwards.
For God's sake, this is not "biases," wtf?? That's straight up just not reading/following the document you're supposed to be testing/reviewing. When I test my documents I actually follow them exactly, step by step, and I always catch these sorts of mistakes. Always. I always copy/paste commands because I know that is what the customer will do, so I have to make sure that works flawlessly.
Dude, if you actually follow your own document, you don't have to pay people to do it for you. Also, you can make sure the person you are paying doesn't ignore the document like you apparently do.
I stopped reading, I'm not interested in whatever else this dude has to say. I'm literally flabbergasted.
Maybe it's because my documents have always gone to real paying customers who have to get through this install, and not some hobby project I'm super proud of or whatever and I don't think I'm super clever? Idk.
> I know someone is going to say "why not just ask an LLM to simulate a range of users?" The answer is very simple - I want to speak to real people. People are brilliant! They can make you laugh, you can see their cat when it wanders on to the call, they bring a unique perspective to the problem, and they're really happy when you give them a €25 voucher. Some will gladly do it for free and make you happy!
So what the author actually paid for was to interact with humans and the README checking was secondary - because, really, my own first thought was literally to ask an LLM check and try to follow the instructions in the README and pretty much any decent LLM (including several local ones) would be able to check if they're adequate and even suggest improvements (just don't let them write it for you :-P).