Can anyone explain to me what this massive volume of code could actually be doing? They obviously have variations of model architectures, training infrastructure, serving infrastructure, and the source for their apps.
Why would the volume that any of this be increasing exponentially? And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?
For one, they have the internal models like Mythos that us normies don't have. Also, machine learning as a field is probably the most automate-able field for AI. It's 100% on computers without "real world" collisions at all, AI can do the whole stack today even with publicly available models. And since it's so empirical, you can make a lot of progress by just running more experiments with more compute.
At some point I imagine they will be writing their own software for the entire training and inference stack. Kernels, libraries, why not? With compute so constrained it must be beneficial to optimize everything as much as possible
I’m not buying this argument, the apps of the LLM providers are some of the most wasteful apps that exist. Have you ever looked at how much memory / CPU Claude and Codex use? They ship with entire virtual machine hypervisors for sandboxing.
I think it’s simply a matter of dogfooding and new feature development, they do have much higher velocity than companies of equivalent size and an unlimited token budget helps as well.
It's AI slop. AI coding agents write much more voluminous code, in my experience. The base code is more expansive, and there are more tests, and there are many, many more commits as you keep fixing and fixing and fixing over and over and over.
I'd bet money they're capturing every change made by any agent, not just at the end of a 'task' (or even a turn). Free training material volume to see what was changed and then what was changed later in the same turn to fix the mistakes introduced.
If they allocate more compute internally than they sell on subs & APIs, then they should be generating more code than all the customers they sell to combined.
> We built Pando with Codex, moving from kick-off to production, in just 10 weeks. The core of Pando was designed by a single engineer, implemented by teams of agents, and deployed by a handful of engineers. Pando began serving production traffic in April of this year, and is one part of a larger system of source control management tools used at OpenAI.
While I don't have any particular interest in the topic this is a great blog post. No AI in every second line, no needless marketing or hype. Just a bunch of engineers talking in their own voice about a topic in great detail with numbers, charts, visualizations.
Another way to avoid having to maximize download speed is to use a global git cache per machine in a similar way that pnpm optimizes installing packages:
> Technical details aside, this volume is ridiculous. How on earth do they see this kind of "growth" as an asset and not a liability?
That coding volume is a liability only for human engineers, with their limited memory, short attention span, tiredness, and quirky coding preferences. LLMs don't have any of those problems.
Not necessarily. High LoC software is a potential bad design smell (without a counterfactual for the same level of functionality it might not be easy to tell though). Badly designed code entering an agent context is always a liability, because it contaminates decisions down the road, and makes new work slower as it needs more tokens to process and navigate old code.
Can anyone explain to me what this massive volume of code could actually be doing? They obviously have variations of model architectures, training infrastructure, serving infrastructure, and the source for their apps.
Why would the volume that any of this be increasing exponentially? And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?
For one, they have the internal models like Mythos that us normies don't have. Also, machine learning as a field is probably the most automate-able field for AI. It's 100% on computers without "real world" collisions at all, AI can do the whole stack today even with publicly available models. And since it's so empirical, you can make a lot of progress by just running more experiments with more compute.
Engineers are working on multiple sessions, and LLM is doing most of the commits/pushes. This is becoming norm at big techs.
If you are wondering what these changes are, you can take a peek at https://github.com/openclaw/openclaw/commits/main/
This isn't to say that such pace of iteration is good or bad. That's a separate discussion.
At some point I imagine they will be writing their own software for the entire training and inference stack. Kernels, libraries, why not? With compute so constrained it must be beneficial to optimize everything as much as possible
I’m not buying this argument, the apps of the LLM providers are some of the most wasteful apps that exist. Have you ever looked at how much memory / CPU Claude and Codex use? They ship with entire virtual machine hypervisors for sandboxing.
I think it’s simply a matter of dogfooding and new feature development, they do have much higher velocity than companies of equivalent size and an unlimited token budget helps as well.
Looking at their desktop apps, I don't think they're into optimizing.
It's the billions of scripts we used not to commit. And billions more.
Maybe it could be faking code and ai usage?
I've seen it on other workplaces, but from what I see from open ai and anthropic, the code shouldn't be as extended.
But maybe I'm crazy lol
It's AI slop. AI coding agents write much more voluminous code, in my experience. The base code is more expansive, and there are more tests, and there are many, many more commits as you keep fixing and fixing and fixing over and over and over.
I'd bet money they're capturing every change made by any agent, not just at the end of a 'task' (or even a turn). Free training material volume to see what was changed and then what was changed later in the same turn to fix the mistakes introduced.
If they allocate more compute internally than they sell on subs & APIs, then they should be generating more code than all the customers they sell to combined.
> We built Pando with Codex, moving from kick-off to production, in just 10 weeks. The core of Pando was designed by a single engineer, implemented by teams of agents, and deployed by a handful of engineers. Pando began serving production traffic in April of this year, and is one part of a larger system of source control management tools used at OpenAI.
Impressive!?
While I don't have any particular interest in the topic this is a great blog post. No AI in every second line, no needless marketing or hype. Just a bunch of engineers talking in their own voice about a topic in great detail with numbers, charts, visualizations.
I don't see benchmarks anywhere in here. For example, how many pushes per second does this handle? And how many concurrent pushers? etc..
Disclaimer: I am building a code forge at https://juju.bi
Another way to avoid having to maximize download speed is to use a global git cache per machine in a similar way that pnpm optimizes installing packages:
https://git-dedup.ben3d.ca
Technical details aside, this volume is ridiculous. How on earth do they see this kind of "growth" as an asset and not a liability?
This is obviously just a response to https://www.anthropic.com/research/redesigning-source-contro.... Probably no one in the chain of writing this even thought about your question.
Instead of keeping it simple, they decided to go stupid.
Because this kind of growth is how they get to RSI.
Whereas previously, creating this amount of code by hand would simply result in RSI.
> Technical details aside, this volume is ridiculous. How on earth do they see this kind of "growth" as an asset and not a liability?
That coding volume is a liability only for human engineers, with their limited memory, short attention span, tiredness, and quirky coding preferences. LLMs don't have any of those problems.
Not necessarily. High LoC software is a potential bad design smell (without a counterfactual for the same level of functionality it might not be easy to tell though). Badly designed code entering an agent context is always a liability, because it contaminates decisions down the road, and makes new work slower as it needs more tokens to process and navigate old code.
Sure, but you also don't know what your LLM is doing without running into these same limitations. So how is this an improvement?
lol nice try openai but its shallow and too late
Impressive. We're definitely seeing LLMs help create real novel software