Wow. Not a Haskell user, but a big user of other languages with expressive type systems (mostly Scala; some Rust). My experience is the complete opposite. I can't imagine using a language without a good type system to catch all the junk the LLM produces. In fact I thought people would move away from languages from poor type systems, like Python, given the cost of using languages with expressive type systems has decreased with LLMs.
exactly, i find the article a wierd take. i would have thougt that being able to catch errors at compile time is the assurance that the LLM generated code is actually decent.
so does this mean that the LLM writes code that is so good that the compiler does not find any more errors?
or is it due to the nature of haskell that makes it hard to write bad code to begin with?
or just that because the haskell compiler catches more errors there is less broken haskell code for the AI to train on?
and what does that mean for the switch to python? if the python compiler/interpreter doesn't catch as many errors do we even know that the code is good?
or is this more like the belief if the LLM can generate good haskell code, surely it can also generate good python?
what's the solution here? speeding up the haskell compiler? if that were easy, would it not already have happened?
personally i still don't trust LLM code generation. i didn't learn haskell yet, but what i hear about it makes me more likely to trust that LLMs can generate good haskell code than python.
i believe the future in LLM code generation is code that can be proven to be correct. proving code correct has been a research topic at some point.
It is a bit surprising, I'd have guessed the same. Although in hindsight I could believe that type systems aren't particularly strong as an anti-bug layer. They help. They're a big boon for coordinating large numbers of mid- and low- skill programmers though because it forces them to go further in documenting their function signatures and makes it much more obvious where the problems are when refactoring spaghetti code because things break loudly.
Refactoring spaghetti has become easier in the LLM era because it can just read all the code, and there is now a skill floor on the programmers that kicks in somewhere relatively high. The benefits of type systems might have suffered because of that.
I think the author largely agrees with you re: type systems and LLMs. He's pretty explicit that Haskell should be very well positioned to be a power language for LLM-assisted programming, but that the Haskell ecosystem presents the bottlenecks that make it harder.
I don't personally use Haskell for anything, but I use Lean and occasionally some other languages with expressive type systems, and like you I've found it to be a pretty great experience for working with LLMs. But I've also experienced what the author is talking about, with languages that sit at different points on the type system spectrum, regarding a languages ecosystem/infra layer becoming a bottleneck. I don't think it's ultimately about the type system but the broader ergonomics of the language/ecosystem.
So I think his criticism is less than expressive type systems are a pre-LLM concept, and more that Haskell has an individually bad "agentic coding story".
IME Python has been very pleasant to use with types, even though they are not nearly as expressive as Haskell. I've noticed a shift in my own work where I spend more time playing with/manipulating change than I do making sure things type check. That does happen, of course, but it happens with less frequency then when I was writing Haskell by hand. During that time, I'd have stack running tests on file change and it was pretty smooth as well, but that workflow breaks down a bit with the current generation of agent harnesses we have.
Recently had to touch a Python project at work. Just setting up the editor needed me to use 2-3 tools out of: pyright, basedpyright, ruff, ty, mypy, and possibly other tools I'm forgetting that kind of do the same thing but throw errors in different parts of the codebase.
Also, for some reason Optional[T] became deprecated, just as the ecosystem finally embraced types ~3 years ago.
In fact, one my company's greenfield projects decided to use TypeScript instead of Python for the [surprisingly] more consistent tooling, and the fact that the big LLM providers all have official TypeScript SDKs anyway. Also, for agentic coding, LLMs don't seem noticeably worse at TypeScript than Python.
My experience can be summarized as:
- for some reason we need 2-3 static analysis tools just for typechecking
- no tool understands each other's comment directives
- each tool reports a different error in your codebase
- even big libraries (e.g. matplotlib) make half their functions return Any
- you'll be tempted to silence the "partially unknown type" warnings, and you'll have to do it for each tool that's running.
> Also, for some reason Optional[T] became deprecated, just as the ecosystem finally embraced types ~3 years ago.
Optional[T] is now T | None. Means exactly the same thing but doesn't require an import. Support for the older syntax presumably won't be removed for a long while regardless.
I grew up on python and after working with Java I really came to appreciate types. However I do want to point out that big libraries like mpl pre-date most efforts for typing, so it is no wonder that they arent typed properly. A lot of these libraries are trying to improve this but it will just take some time.
I'm pretty sure that's the general trend and it will continue.
But I do think what benefits LLMs is the speed and accuracy of feedback. Type systems cover the accuracy part, but haskell was killing them on speed. It seems like a strange choice to go so far the other way on accuracy when there's a lot of languages in between. But I'm not familiar with the project so not in a position to call it.
It's not also really about expressiveness IMO. I've found LLMs to be best with more constrained type systems: they are better at ocaml than they are at typescript.
Java can also have 15+ minute cold compiles on large projects if you kill all caches. It's less bad on smaller codebases because you don't have to recompile dependencies if you target a bytecode vm, but if you always gate feedback on a cold compile in a fresh VM you just aren't gonna beat an interpreted language
But I'd look at people a bit oddly if they said: 'We didn't want to set up CI caching and compiled languages took 30 minutes per run so we changed our entire codebase to python'.
Maybe it makes sense for them, and caching across dynamically spawned VM's is admittedly a harder problem which most build systems aren't great at, but still. I can easily believe that getting build caching to be reliable would be a lot of work, but is it more work than a full rewrite of a significant codebase?
> I've found LLMs to be best with more constrained type systems: they are better at ocaml than they are at typescript.
When the potential set of behaviors you could write a program to have is infinite, but the actual behavior you want is singular, a programming language is more importantly defined by which ones it eliminates up front than which ones it lets you write (assuming it lets you write the one you want at all, but that's almost always going to be the case for most general purpose languages). Bugs are just false positives in this framing, where the program you wrote seems like the one you wanted, but there's some divergence between what you thought you were getting and what you actually got, and catching some of those up front is a huge part of why type systems are so useful.
I strongly agree with the premise of this article, which is why I am surprised that the author moved away from Haskell to Python.
For some time now it’s felt clear (or at least extremely) compelling that agents need fast compile times in order to be effective, especially when you’re working in parallel. But the other thing that has felt just as obvious is that agents need strong type systems and narrow guardrails in order to constrain their outputs. These two things felt clear enough to me that, like the author, I wanted to choose a language ecosystem that maximized them. There _are_ languages that both have expressive type systems _and_ fast compile times. I wonder if the author investigated any of them, before deciding that no compilation time at all was acceptable.
In my case I landed in OCaml. I think there are other options in the space—Go if you want less typing but faster compiles; Rust if you want more types but slower compiles. My mostly vibes-based evaluation landed on OCaml, and I’ve been pretty happy with the results.
>agents need strong type systems and narrow guardrails
I read the second paragraph of linked article as saying close to the opposite of that, particularly,
"the model can often avoid the mistake before the compiler ever sees the code. And as the models get better, the relative value of catching every possible issue at compile time changes."
In other words, LLMs are much less likely than humans to make dumb, fat-finger mistakes, and, when they do, are able to catch and fix them more quickly, ergo the value of type checking has fallen.
Everything in the prior sentence is, obviously, highly debatable. But it felt like part of the premise.
I've gotten best results with LLMs generating Go, Java, and C# code as they have the best combination of strong type systems and fast or no compile times.
I'm not trying to be reductive but the article's a lot of words for "We're vibecoding our app now and the glorious (almost almighty) Haskell compiler is too slow for the agent to iterate it's mistakes until it gets it right."
This thought completely neglects the idea that Haskell probably needs significantly less compiler runs because every run catches more errors and gives more information about them.
And that is not even considering how often the agent needs to run tests to get it right.
I'm not a Haskell developer and I hadn't heard of this company "Scarf" before.
As much as I respect this guy who tried to work and push an alternative ecosystem, it's hard for me to shake off the impression that, rather than due to Haskell compile time, he moved to python because it's easier to find developers for it and it's the de facto scripting language for LLMs.
No problem about that, of course. Running a company is hard enough, I think that passion and idealism for a language/platform/technology out of aesthetic appreciation can only go so far and after a certain age just making money and reaching your professional objectives count more.
Fast compile times is one of the most important qualities for developer productivity. It made Haskell a non-starter for many developers even before LLM driven development took off.
Very surprised about this decision. I use strict languages much more with LLMs and they improve quality a lot. Working on a big python codebase is very painful with an LLM.
This is a good post. AI has changed the programming language trade-offs and, as someone running a company that uses both Haskell and Python, I hope that Haskell can adapt to this new era.
I would like to add one additional observation, since we have been using both Haskell and Python in production for a long time:
Haskell excels at platform work, while Python excels at product work.
Our infrastructure teams work in Haskell (and also Rust nowadays), while our product teams work in Python. This gives us the best of both worlds (in my opinion): fast and rock-solid infrastructure on the platform side, and fast development speed and quick iteration cycles on the product side.
This setup has worked well for years for us, but it remains to be seen how and if this is going to change as well in the new AI era.
For what it's worth, I've been using Haskell in production at Bitnomial, a financial exchange, and LLMs + Haskell is an extremely productive combo.
Since Opus 4.6, LLMs have been pretty clever at using fancy types with libraries like Servant and Beam. The expressiveness of the types, combined with feedback from the compiler, means that agents converge quickly to something that works. I don't think I've noticed agents having to run the compiler so often that compilation speed is an issue.
I like this article, but I would take some issue with the concept of the percentage of time taken up being a major issue.
If you go from taking 2 days to write some code and 20 minutes to type check (which does seem long, don't get me wrong, but still) to 10 minutes to prompt some code and 20 minutes to type check, that percentage increase to me isn't enough to justify switching.
You're still almost 2 days ahead, and converting those 20 minutes to 20 seconds are not going to make you ship a feature appreciably faster. But those types stand strong and I don't believe they can yet be replaced by an LLM believing they're correct.
Having said that, I also think that Haskell should massively speed things up. Having strong types if nothing else should surely produce some amazing type-checking speed wins.
I've been a power user of LLMs for software development for a while now, and I've found two things to be true:
- The benefits of more "extreme" type systems are more accessible and valuable than ever. I have a fairly involved project built on Lean that I hope to open source this month, and it's been a joy to work in even for uses outside of mathematics.
- Readability, build time, infra complexity, and everything that affects your speed after finishing your implementation--these things now matter more than ever.
It's sort of a dual ergonomics problem, in some sense. And given that, the author's lament makes complete sense to me, especially:
"An AI-enabled Haskell ecosystem would ask different questions. How do we make Haskell easier for agents to use well? How do we get more high-quality Haskell examples into model training data? How can we scale reviews? How do we make library docs full of copy-pastable, realistic examples, not just beautiful types? How do we make project bootstrap fast? How do we make error messages more agent-friendly? How do we reduce cold build times? How do we make common industrial patterns obvious to a model that is trying to help?"
I am increasingly wondering if we are in a post-language world in terms of development. Why would I ask an agent to write a server in anything other than the most efficient language, although efficiency can take several forms: runtime, token usage during development, and wall clock dev time (affected by slow compile times for example).
My intuition is that type-safe languages with fast compilers are the best option. Maybe Go? I personally prefer Java just due to my experience running it in production, but am not sure there's many arguments for it over Go in a greenfield application. The other candidate would be Rust, but I worry about token efficiency and tool performance, I suspect it's not worth it for the runtime improvements.
All that being said, in this article switching to Python seems like a wild choice. Relatively poor performance, no compile time checking at all. Python's big selling point was developer ergonomics, which seems largely irrelevant now.
These are all just thoughts at the moment, I should try to find some evidence one way or another.
My thought reading this article was: Why write the system in any one language at all? And I don't just mean having some parts in one language and others in another language, I mean redundant implementations of the same parts. You can use an AI to rewrite parts of the system, and then throw away the old part... or you could just keep the old part.
That is: Have a Haskell base system. Have a Python "development" version on which you iterate at lightning speed. But also, in the background, moving at whatever pace it takes, have an agent running that imports all the Python development changes into the Haskell version. Have nightly builds of the Haskell version to reap its benefits (issues caught by the type system, more efficient native code).
And it doesn't have to be Python/Haskell of course. The "development" version could be a (hypothetical?) interpreted Haskell. I have no idea if ghci would be useful for this. Neither do I know if the 15-minute Haskell build time is spent in the frontend (so an interpreter would have to pay that cost too) or in code generation or linking (which the interpreter wouldn't need to care about). Anyway, these are things I would think about before I do what the OP did.
Language choice had less impact than people first assume even before LLMs in most software. A good engineering team produces good code in whatever language they happen to be using. In my own career I've worked in serious Java, Scala, Haskell, Javascript, PHP, and Python application stacks and I've seen plenty of good and bad examples.
I reckon language choice matters more at the edges of economic activity where a specific language feature really does make the difference in the end product, but most activity that is leveraging LLMs now is more generic enterprise SaaS software.
You are ignoring LLM-ergonomics, some time ago I saw benchmarks showing that popularity the language (and so more data available in training date) was strongly linked with LLM's performances, with top results with javascript and python. I don't know if a year later this is still true, but is absolutely possible
There is one simple thing you have to realize why Python is the optimal choice. You have so much training data. Python is the second most popular language on GitHub and is easy to read.
We at Cachix have also moved on from Haskell about two years ago and I'm sure someone is going to make a comeback with a language that takes the lessons from it but starts from skratch.
We need more general purpose Elm languages in the space.
I'm curious about the choice of Python, rather than TypeScript.
I find Ruby a very beautiful language, and Rails is an excellent web framework, but I need typed functions, record types and sum types.
They help not just with correctness, but also as living documentation that lets me understand AI generated code. TypeScript provides discriminated union, but not exhaustive pattern-matching, and its syntax is a bit verbose, but since I'm no longer writing most of the code myself, I can live with it.
However I can't imagine using Python or any other dynamic language going forward. There is likely good reason for you to choose it, and I'm curious to know what that is.
Yes, because the code you (or the LLM) writes is only part of the equation; if you use third party libraries then it becomes an ecosystem problem. I'm not currently using much Python but my understanding is that the community has not yet aligned on typing nearly as well as the TS community has.
I hadn't considered that, you are right. Though I would still prefer TS because the language is all about types; the entire ecosystem is well-typed and the type system is quite powerful (I do enjoy an occasional Omit and Pick). But it is a personal choice; as long as LLMs can generate well-typed Python, then for people who like the language it makes sense. However from the article, I got the sense that they went completely dynamic.
This is quite insane to me. If I compare the output of LLMs for python vs statically typed languages it's really not a good choice to go the python route. It consistently produce relatively garbage code along actually good code. My experience has that the better static typing you have the better the code becomes.
LLMs have made me move away more from python rather than into it. I'm very surprised by this experiences of the author. The article is all over the place as well. Going basically all in on Python because it is apparently better than Haskell for LLM use and than agreeing with someone that says Rust is the best.
Languages designers will have to make a choice whether to continue to design for humans or for big slop machines. The design goals are not compatible. This is obvious. I don't understand how anyone can miss such an obvious point.
Another obvious point is that an industry that runs on code slop will stagnate in terms of language an human tooling design.
"Hammers are now a very popular tool, and one can move quickly building exclusively with hammers, so we have decided to construct buildings strictly using nails, no more screws, bolts, or any other kind of fastener shall be used going forward."
Wow. Not a Haskell user, but a big user of other languages with expressive type systems (mostly Scala; some Rust). My experience is the complete opposite. I can't imagine using a language without a good type system to catch all the junk the LLM produces. In fact I thought people would move away from languages from poor type systems, like Python, given the cost of using languages with expressive type systems has decreased with LLMs.
exactly, i find the article a wierd take. i would have thougt that being able to catch errors at compile time is the assurance that the LLM generated code is actually decent.
so does this mean that the LLM writes code that is so good that the compiler does not find any more errors?
or is it due to the nature of haskell that makes it hard to write bad code to begin with?
or just that because the haskell compiler catches more errors there is less broken haskell code for the AI to train on?
and what does that mean for the switch to python? if the python compiler/interpreter doesn't catch as many errors do we even know that the code is good?
or is this more like the belief if the LLM can generate good haskell code, surely it can also generate good python?
what's the solution here? speeding up the haskell compiler? if that were easy, would it not already have happened?
personally i still don't trust LLM code generation. i didn't learn haskell yet, but what i hear about it makes me more likely to trust that LLMs can generate good haskell code than python.
i believe the future in LLM code generation is code that can be proven to be correct. proving code correct has been a research topic at some point.
It is a bit surprising, I'd have guessed the same. Although in hindsight I could believe that type systems aren't particularly strong as an anti-bug layer. They help. They're a big boon for coordinating large numbers of mid- and low- skill programmers though because it forces them to go further in documenting their function signatures and makes it much more obvious where the problems are when refactoring spaghetti code because things break loudly.
Refactoring spaghetti has become easier in the LLM era because it can just read all the code, and there is now a skill floor on the programmers that kicks in somewhere relatively high. The benefits of type systems might have suffered because of that.
I think the author largely agrees with you re: type systems and LLMs. He's pretty explicit that Haskell should be very well positioned to be a power language for LLM-assisted programming, but that the Haskell ecosystem presents the bottlenecks that make it harder.
I don't personally use Haskell for anything, but I use Lean and occasionally some other languages with expressive type systems, and like you I've found it to be a pretty great experience for working with LLMs. But I've also experienced what the author is talking about, with languages that sit at different points on the type system spectrum, regarding a languages ecosystem/infra layer becoming a bottleneck. I don't think it's ultimately about the type system but the broader ergonomics of the language/ecosystem.
So I think his criticism is less than expressive type systems are a pre-LLM concept, and more that Haskell has an individually bad "agentic coding story".
IME Python has been very pleasant to use with types, even though they are not nearly as expressive as Haskell. I've noticed a shift in my own work where I spend more time playing with/manipulating change than I do making sure things type check. That does happen, of course, but it happens with less frequency then when I was writing Haskell by hand. During that time, I'd have stack running tests on file change and it was pretty smooth as well, but that workflow breaks down a bit with the current generation of agent harnesses we have.
Recently had to touch a Python project at work. Just setting up the editor needed me to use 2-3 tools out of: pyright, basedpyright, ruff, ty, mypy, and possibly other tools I'm forgetting that kind of do the same thing but throw errors in different parts of the codebase.
Also, for some reason Optional[T] became deprecated, just as the ecosystem finally embraced types ~3 years ago.
In fact, one my company's greenfield projects decided to use TypeScript instead of Python for the [surprisingly] more consistent tooling, and the fact that the big LLM providers all have official TypeScript SDKs anyway. Also, for agentic coding, LLMs don't seem noticeably worse at TypeScript than Python.
My experience can be summarized as:
- for some reason we need 2-3 static analysis tools just for typechecking
- no tool understands each other's comment directives
- each tool reports a different error in your codebase
- even big libraries (e.g. matplotlib) make half their functions return Any
- you'll be tempted to silence the "partially unknown type" warnings, and you'll have to do it for each tool that's running.
> Also, for some reason Optional[T] became deprecated, just as the ecosystem finally embraced types ~3 years ago.
Optional[T] is now T | None. Means exactly the same thing but doesn't require an import. Support for the older syntax presumably won't be removed for a long while regardless.
I grew up on python and after working with Java I really came to appreciate types. However I do want to point out that big libraries like mpl pre-date most efforts for typing, so it is no wonder that they arent typed properly. A lot of these libraries are trying to improve this but it will just take some time.
TFA:
The type safety we gave up hasn’t been noticeable in any concrete way yet, especially considering our test coverage has never been better.
> I can't imagine using a language without a good type system to catch all the junk the LLM produces
One approach would be to not use LLMs.
I'm pretty sure that's the general trend and it will continue.
But I do think what benefits LLMs is the speed and accuracy of feedback. Type systems cover the accuracy part, but haskell was killing them on speed. It seems like a strange choice to go so far the other way on accuracy when there's a lot of languages in between. But I'm not familiar with the project so not in a position to call it.
It's not also really about expressiveness IMO. I've found LLMs to be best with more constrained type systems: they are better at ocaml than they are at typescript.
Java can also have 15+ minute cold compiles on large projects if you kill all caches. It's less bad on smaller codebases because you don't have to recompile dependencies if you target a bytecode vm, but if you always gate feedback on a cold compile in a fresh VM you just aren't gonna beat an interpreted language
But I'd look at people a bit oddly if they said: 'We didn't want to set up CI caching and compiled languages took 30 minutes per run so we changed our entire codebase to python'.
Maybe it makes sense for them, and caching across dynamically spawned VM's is admittedly a harder problem which most build systems aren't great at, but still. I can easily believe that getting build caching to be reliable would be a lot of work, but is it more work than a full rewrite of a significant codebase?
Additionally in modern Java there are even the options of AOT and JIT caches, which can be reused across runs.
Or if staying on Linux, JVM snapshots.
> I've found LLMs to be best with more constrained type systems: they are better at ocaml than they are at typescript.
When the potential set of behaviors you could write a program to have is infinite, but the actual behavior you want is singular, a programming language is more importantly defined by which ones it eliminates up front than which ones it lets you write (assuming it lets you write the one you want at all, but that's almost always going to be the case for most general purpose languages). Bugs are just false positives in this framing, where the program you wrote seems like the one you wanted, but there's some divergence between what you thought you were getting and what you actually got, and catching some of those up front is a huge part of why type systems are so useful.
You should read the article before replying.
I strongly agree with the premise of this article, which is why I am surprised that the author moved away from Haskell to Python.
For some time now it’s felt clear (or at least extremely) compelling that agents need fast compile times in order to be effective, especially when you’re working in parallel. But the other thing that has felt just as obvious is that agents need strong type systems and narrow guardrails in order to constrain their outputs. These two things felt clear enough to me that, like the author, I wanted to choose a language ecosystem that maximized them. There _are_ languages that both have expressive type systems _and_ fast compile times. I wonder if the author investigated any of them, before deciding that no compilation time at all was acceptable.
In my case I landed in OCaml. I think there are other options in the space—Go if you want less typing but faster compiles; Rust if you want more types but slower compiles. My mostly vibes-based evaluation landed on OCaml, and I’ve been pretty happy with the results.
>agents need strong type systems and narrow guardrails
I read the second paragraph of linked article as saying close to the opposite of that, particularly,
"the model can often avoid the mistake before the compiler ever sees the code. And as the models get better, the relative value of catching every possible issue at compile time changes."
In other words, LLMs are much less likely than humans to make dumb, fat-finger mistakes, and, when they do, are able to catch and fix them more quickly, ergo the value of type checking has fallen.
Everything in the prior sentence is, obviously, highly debatable. But it felt like part of the premise.
I've gotten best results with LLMs generating Go, Java, and C# code as they have the best combination of strong type systems and fast or no compile times.
Yes, and in Java/C# case, AOT compilation is also available.
I would also add Kotlin, Clojure and F#.
Scala not really as the compilation is not much better, and since the Scala 3 reboot, the ecosystem doesn't seem to be doing that well.
The market opportunity for Haskell on the JVM is gone, although they are doing cool stuff with capabilities.
I'm not trying to be reductive but the article's a lot of words for "We're vibecoding our app now and the glorious (almost almighty) Haskell compiler is too slow for the agent to iterate it's mistakes until it gets it right."
This thought completely neglects the idea that Haskell probably needs significantly less compiler runs because every run catches more errors and gives more information about them.
And that is not even considering how often the agent needs to run tests to get it right.
Maybe overly verbose but makes an important point.
Slow compile times should have been a deal breaker for how they impacted human coders. LLM coding just makes the problem more stark.
I'm not a Haskell developer and I hadn't heard of this company "Scarf" before.
As much as I respect this guy who tried to work and push an alternative ecosystem, it's hard for me to shake off the impression that, rather than due to Haskell compile time, he moved to python because it's easier to find developers for it and it's the de facto scripting language for LLMs.
No problem about that, of course. Running a company is hard enough, I think that passion and idealism for a language/platform/technology out of aesthetic appreciation can only go so far and after a certain age just making money and reaching your professional objectives count more.
How the hell would you know his true intentions?
Fast compile times is one of the most important qualities for developer productivity. It made Haskell a non-starter for many developers even before LLM driven development took off.
Very surprised about this decision. I use strict languages much more with LLMs and they improve quality a lot. Working on a big python codebase is very painful with an LLM.
This is a good post. AI has changed the programming language trade-offs and, as someone running a company that uses both Haskell and Python, I hope that Haskell can adapt to this new era.
I would like to add one additional observation, since we have been using both Haskell and Python in production for a long time:
Haskell excels at platform work, while Python excels at product work.
Our infrastructure teams work in Haskell (and also Rust nowadays), while our product teams work in Python. This gives us the best of both worlds (in my opinion): fast and rock-solid infrastructure on the platform side, and fast development speed and quick iteration cycles on the product side.
This setup has worked well for years for us, but it remains to be seen how and if this is going to change as well in the new AI era.
For what it's worth, I've been using Haskell in production at Bitnomial, a financial exchange, and LLMs + Haskell is an extremely productive combo.
Since Opus 4.6, LLMs have been pretty clever at using fancy types with libraries like Servant and Beam. The expressiveness of the types, combined with feedback from the compiler, means that agents converge quickly to something that works. I don't think I've noticed agents having to run the compiler so often that compilation speed is an issue.
I like this article, but I would take some issue with the concept of the percentage of time taken up being a major issue.
If you go from taking 2 days to write some code and 20 minutes to type check (which does seem long, don't get me wrong, but still) to 10 minutes to prompt some code and 20 minutes to type check, that percentage increase to me isn't enough to justify switching.
You're still almost 2 days ahead, and converting those 20 minutes to 20 seconds are not going to make you ship a feature appreciably faster. But those types stand strong and I don't believe they can yet be replaced by an LLM believing they're correct.
Having said that, I also think that Haskell should massively speed things up. Having strong types if nothing else should surely produce some amazing type-checking speed wins.
> The type system caught real bugs.
> The model can often avoid the mistake before the compiler ever sees the code.
> The type safety we gave up hasn’t been noticeable in any concrete way yet [...]
> Type safety can be a huge advantage for LLM-generated code if the compiler is helping the agent converge quickly.
Well, good to have this question cleared up once and for all :-)
I've been a power user of LLMs for software development for a while now, and I've found two things to be true:
- The benefits of more "extreme" type systems are more accessible and valuable than ever. I have a fairly involved project built on Lean that I hope to open source this month, and it's been a joy to work in even for uses outside of mathematics.
- Readability, build time, infra complexity, and everything that affects your speed after finishing your implementation--these things now matter more than ever.
It's sort of a dual ergonomics problem, in some sense. And given that, the author's lament makes complete sense to me, especially:
"An AI-enabled Haskell ecosystem would ask different questions. How do we make Haskell easier for agents to use well? How do we get more high-quality Haskell examples into model training data? How can we scale reviews? How do we make library docs full of copy-pastable, realistic examples, not just beautiful types? How do we make project bootstrap fast? How do we make error messages more agent-friendly? How do we reduce cold build times? How do we make common industrial patterns obvious to a model that is trying to help?"
I am increasingly wondering if we are in a post-language world in terms of development. Why would I ask an agent to write a server in anything other than the most efficient language, although efficiency can take several forms: runtime, token usage during development, and wall clock dev time (affected by slow compile times for example).
My intuition is that type-safe languages with fast compilers are the best option. Maybe Go? I personally prefer Java just due to my experience running it in production, but am not sure there's many arguments for it over Go in a greenfield application. The other candidate would be Rust, but I worry about token efficiency and tool performance, I suspect it's not worth it for the runtime improvements.
All that being said, in this article switching to Python seems like a wild choice. Relatively poor performance, no compile time checking at all. Python's big selling point was developer ergonomics, which seems largely irrelevant now.
These are all just thoughts at the moment, I should try to find some evidence one way or another.
My thought reading this article was: Why write the system in any one language at all? And I don't just mean having some parts in one language and others in another language, I mean redundant implementations of the same parts. You can use an AI to rewrite parts of the system, and then throw away the old part... or you could just keep the old part.
That is: Have a Haskell base system. Have a Python "development" version on which you iterate at lightning speed. But also, in the background, moving at whatever pace it takes, have an agent running that imports all the Python development changes into the Haskell version. Have nightly builds of the Haskell version to reap its benefits (issues caught by the type system, more efficient native code).
And it doesn't have to be Python/Haskell of course. The "development" version could be a (hypothetical?) interpreted Haskell. I have no idea if ghci would be useful for this. Neither do I know if the 15-minute Haskell build time is spent in the frontend (so an interpreter would have to pay that cost too) or in code generation or linking (which the interpreter wouldn't need to care about). Anyway, these are things I would think about before I do what the OP did.
Language choice had less impact than people first assume even before LLMs in most software. A good engineering team produces good code in whatever language they happen to be using. In my own career I've worked in serious Java, Scala, Haskell, Javascript, PHP, and Python application stacks and I've seen plenty of good and bad examples.
I reckon language choice matters more at the edges of economic activity where a specific language feature really does make the difference in the end product, but most activity that is leveraging LLMs now is more generic enterprise SaaS software.
You are ignoring LLM-ergonomics, some time ago I saw benchmarks showing that popularity the language (and so more data available in training date) was strongly linked with LLM's performances, with top results with javascript and python. I don't know if a year later this is still true, but is absolutely possible
There is one simple thing you have to realize why Python is the optimal choice. You have so much training data. Python is the second most popular language on GitHub and is easy to read.
We at Cachix have also moved on from Haskell about two years ago and I'm sure someone is going to make a comeback with a language that takes the lessons from it but starts from skratch.
We need more general purpose Elm languages in the space.
Isn’t tgat language Rust, practically speaking?
I would love to see that happen!
what did you move to, and why?
Rust, mostly because:
- haskell exceptions and laziness are devastating for production
- too small ecosystem, had to write 10+ SDKs (now with AI that's less of an issue)
- haskell ecosystem is too fragmented due to prima donna prevalence
I'm curious about the choice of Python, rather than TypeScript.
I find Ruby a very beautiful language, and Rails is an excellent web framework, but I need typed functions, record types and sum types.
They help not just with correctness, but also as living documentation that lets me understand AI generated code. TypeScript provides discriminated union, but not exhaustive pattern-matching, and its syntax is a bit verbose, but since I'm no longer writing most of the code myself, I can live with it.
However I can't imagine using Python or any other dynamic language going forward. There is likely good reason for you to choose it, and I'm curious to know what that is.
python has optional types too now. if you could get the LLM to produce typed python would it be any worse than typescript?
Yes, because the code you (or the LLM) writes is only part of the equation; if you use third party libraries then it becomes an ecosystem problem. I'm not currently using much Python but my understanding is that the community has not yet aligned on typing nearly as well as the TS community has.
my understanding is that the [python] community has not yet aligned on typing nearly as well as the TS community has
this is undoubtedly true.
I hadn't considered that, you are right. Though I would still prefer TS because the language is all about types; the entire ecosystem is well-typed and the type system is quite powerful (I do enjoy an occasional Omit and Pick). But it is a personal choice; as long as LLMs can generate well-typed Python, then for people who like the language it makes sense. However from the article, I got the sense that they went completely dynamic.
Note that Scarf is not moving away from Haskell for performance-sensitive applications [0], which was the immediate question I had reading this.
[0]: https://discourse.haskell.org/t/after-7-years-in-production-...
I was expecting yet another moving from Haskell into Rust article, instead they went to Python.
Who cares about performance.
This is quite insane to me. If I compare the output of LLMs for python vs statically typed languages it's really not a good choice to go the python route. It consistently produce relatively garbage code along actually good code. My experience has that the better static typing you have the better the code becomes.
LLMs have made me move away more from python rather than into it. I'm very surprised by this experiences of the author. The article is all over the place as well. Going basically all in on Python because it is apparently better than Haskell for LLM use and than agreeing with someone that says Rust is the best.
Languages designers will have to make a choice whether to continue to design for humans or for big slop machines. The design goals are not compatible. This is obvious. I don't understand how anyone can miss such an obvious point.
Another obvious point is that an industry that runs on code slop will stagnate in terms of language an human tooling design.
Robots need formal specification languages, to tame non deterministic compilers.
I am absolutely baffled at the idea that LLMs mean you need less automated verification of correctness.
Wow, another bunch of people who give up engineering to satisfy their addiction to speed.
"Hammers are now a very popular tool, and one can move quickly building exclusively with hammers, so we have decided to construct buildings strictly using nails, no more screws, bolts, or any other kind of fastener shall be used going forward."