GitHub, including Enterprise (so not just Azure’s fault), has become unreliable, low quality software.
I say unreliable because I cannot rely on it to accurately do what it claims to do. I cannot trust any number. I cannot trust that actions I invoke will actually happen. I cannot trust that taking action will not have mysterious side effects (e.g. re-opening an issue my boss’s boss closed in 2017 without explanation). I think I trust the underlying git infra but at this point I’m not sure why I do.
I say low quality because it is extremely slow at random moments, gets into broken inconsistent UI states, has baffling UX choices that make it harder to navigate than it should be (particularly with new UIs), and overall just doesn’t have the fit and finish we should expect from software in 1996 - or 2026, or literally any time in between.
For months now I've had scheduled GH Actions workflows that don't run on time, sometimes not at all. Like we're talking 4-8 hours later than the cron statement. Yesterday I finally gave up and just moved the trigger to AWS EventBridge Scheduler/Lambda. Maddening, but also we're talking code repos that are $0 for me. Not production level. I can't imagine people running Production pipelines on github.com.
Dirty reads is a classic symptom of scaling poorly. Eventually consistent database writes are fine, but it's a bad look if a user takes an action and then doesn't see the results of their action.
Similar to how every description of relaxed consistency for CPUs starts out by pointing out your own reads always observe your own writes in program order, as if to say, "don't worry, we're not insane."
The fact that there's so much indirection between what happens in the registers and memory in the machine, the underlying reality being modeled by software, and the information displayed semantically to the users, is a travesty.
Software could be so much simpler and more reliable. There's so much bloat that is necessary to solve problems created by bloat. I hope we find our way out of this mess sooner rather than later. I'd hate for entire generations to suffer the current state of software whereas the theoretical understanding necessary to make things better was produced very early in the history of programmable computers.
There's tonnes of UI bugs. A more severe one I've experienced is that my approved PRs have been shown as still waiting on review, leading some of them to be delayed by weeks.
Calling this "severe" feels overblown. It's a UI mismatch likely caused by using multiple caches that are out of sync. I feel like every engineer on this site has encountered this exact bug at one point or another. It's an easy one to run into, especially in bigger orgs.
This got to be the extinguish phase from the EEE Microsoft playbook. Prior acquisition, Github was very liked and very focus, then Microsoft happened in their embrace phase, telling us EEE was something of the past. While trying to show good faith they extended the platform and this is now the very last phase
Presumably you realize most current Microsoft employees were not even alive when the EEE memo was written.
Must we also talk about some decision Henry Ford made in 1906 every time the automaker is mentioned, or something the Gauls did in 200 BC every time England is mentioned?
You don't want to talk about it because either 1) you don't know it, or 2) you were downplaying it at the time, too.
The only time we talk about Henry Ford, unless it's about his anti-Semitism, is about decisions he made in 1906. If we go by your arbitrary standard, why would we talk about Henry Ford at all when we could talk about this year's Tesla model?
> See also Netscape v2.0 and 3.0, which were badass browsers back in 1996, but are similarly irrelevant today.
Why are you talking about Netscape? What does that have to do with anything happening right now?
I don't believe this is true of the past, but it'd be quite the twist if the extinguish phase was really just pure incompetence this whole time. It certainly seems that way right now with GitHub.
GitHub, including Enterprise (so not just Azure’s fault), has become unreliable, low quality software.
I say unreliable because I cannot rely on it to accurately do what it claims to do. I cannot trust any number. I cannot trust that actions I invoke will actually happen. I cannot trust that taking action will not have mysterious side effects (e.g. re-opening an issue my boss’s boss closed in 2017 without explanation). I think I trust the underlying git infra but at this point I’m not sure why I do.
I say low quality because it is extremely slow at random moments, gets into broken inconsistent UI states, has baffling UX choices that make it harder to navigate than it should be (particularly with new UIs), and overall just doesn’t have the fit and finish we should expect from software in 1996 - or 2026, or literally any time in between.
For months now I've had scheduled GH Actions workflows that don't run on time, sometimes not at all. Like we're talking 4-8 hours later than the cron statement. Yesterday I finally gave up and just moved the trigger to AWS EventBridge Scheduler/Lambda. Maddening, but also we're talking code repos that are $0 for me. Not production level. I can't imagine people running Production pipelines on github.com.
Some of the oldest PRs could no longer be found.
For example, when I decided to continue my eight-year-old PR https://github.com/ClickHouse/ClickHouse/pull/104948, I couldn't see it - neither in search nor while navigating through pages.
Direct links still work.
Dirty reads is a classic symptom of scaling poorly. Eventually consistent database writes are fine, but it's a bad look if a user takes an action and then doesn't see the results of their action.
Similar to how every description of relaxed consistency for CPUs starts out by pointing out your own reads always observe your own writes in program order, as if to say, "don't worry, we're not insane."
The fact that there's so much indirection between what happens in the registers and memory in the machine, the underlying reality being modeled by software, and the information displayed semantically to the users, is a travesty.
Software could be so much simpler and more reliable. There's so much bloat that is necessary to solve problems created by bloat. I hope we find our way out of this mess sooner rather than later. I'd hate for entire generations to suffer the current state of software whereas the theoretical understanding necessary to make things better was produced very early in the history of programmable computers.
I think they struggle with two things
1) counting
2) cache validation
3) off-by-one errors
According to the documentation there are four things:
4) naming things
5) keeping the documentation up-to-date
There are solutions: https://xkcd.com/3062/
4) naming things
"Since we forced our engineers to agentic workflows our efficiency doubled. Just look at the numbers!"
> Really curious as to what’s going on over there, if you’ve got any insight let me know!
Microsoft. That's the answer. Microsoft is happening over there.
There's tonnes of UI bugs. A more severe one I've experienced is that my approved PRs have been shown as still waiting on review, leading some of them to be delayed by weeks.
I'm partial to the one where the workflow completes but the its summary on the PR does not get updated, so it appears to be running.
How many companies engender taste in bugs, eh??
I have got an insight alright. Microsoft took it over.
My phone consistently says that I have one more update than they'll actually let me download... what that one phantom app is, I'll never know...
CIA Information Services.
Calling this "severe" feels overblown. It's a UI mismatch likely caused by using multiple caches that are out of sync. I feel like every engineer on this site has encountered this exact bug at one point or another. It's an easy one to run into, especially in bigger orgs.
The second screenshot with “1 3 Next” seems pretty hard to justify.
Taking care that such incongruent representations can’t happen even in the face of inconsistent backend caches is part of proper UI state management.
Yes but in the case of GH, it also happens in cases where the data is old: not just a temporary drift.
AI will solve this
Press "Next"
This got to be the extinguish phase from the EEE Microsoft playbook. Prior acquisition, Github was very liked and very focus, then Microsoft happened in their embrace phase, telling us EEE was something of the past. While trying to show good faith they extended the platform and this is now the very last phase
>EEE
My god what an ancient trope.
Presumably you realize most current Microsoft employees were not even alive when the EEE memo was written.
Must we also talk about some decision Henry Ford made in 1906 every time the automaker is mentioned, or something the Gauls did in 200 BC every time England is mentioned?
You don't want to talk about it because either 1) you don't know it, or 2) you were downplaying it at the time, too.
The only time we talk about Henry Ford, unless it's about his anti-Semitism, is about decisions he made in 1906. If we go by your arbitrary standard, why would we talk about Henry Ford at all when we could talk about this year's Tesla model?
> See also Netscape v2.0 and 3.0, which were badass browsers back in 1996, but are similarly irrelevant today.
Why are you talking about Netscape? What does that have to do with anything happening right now?
Maybe, but does Team Foundation Server even exist anymore? What is the MS branded service that degrading GitHub is supposed to drive people to?
Copilot, of course. Helpfully, every Microslop product is named Copilot Something now. Take your pick.
I don't believe this is true of the past, but it'd be quite the twist if the extinguish phase was really just pure incompetence this whole time. It certainly seems that way right now with GitHub.
rails + russian doll style caching + large engineering team + azure as a platform = recipe for this exact problem.