As someone that has used Dagger a lot (a previous daggernaut / ambassador dropped off after LLMs was announced, and was changing jobs at the time. implemented it at a previous company across 95% of services, built the rust sdk) the approach was and is amazing for building complex build chains.
It serves a place where a dockerfile is not enough, and CI workflows are too difficult to debug or reason about.
I do have some current problems with it though:
1. I don't care at all about the LLM agent workflows, I get that it is possible, but the same people that chose dagger for what it was, is not the same audience that runs agents like that. I can't choose dagger currently, because I don't know if they align with my interests as an engineer solving a specific problems for where I work (delivering software, not running agents).
2. I advocated for modules before it was a thing, but I never implemented it. It is too much magic, I want to write code, not a DSL that looks like code, dagger is already special in that regard, to modules takes it a step too far. You can't find the code in their docs anymore, but dagger can be written with just a .go, .py or .rs file. Simply take in dagger as a dependency and build your workflow.
3. Too complex to operate, dagger doesn't have runners currently, and it is difficult to run a setup in production for CI yourself, without running it in the actions themselves, which can be disastrous for build times, as dagger often leads you into using quite a few images, so having a cache is a must.
Dagger needs to choose and execute; not having runners, even when we we're willing to throw money at them was a mistake IMO. Love the tool, the team, the vision but it is too distracted, magical and impatient to pick up at the moment.
Hi Kasper, good to see you here! Thank you for the detailed feedback.
1. Yes we got over-excited with the agent runtime use case. We stand by the LLM implementation because we never compromised on the integrity of Dagger's modular design. But our marketing and product priorities were all over the place. We're going to refocus on the original use case: helping you ship software, and more particularly building & testing it.
2. Modules have warts but they are essential. We will continue to improve them, and remain committed to them. Without this feature, you have to write a complete standalone program every time you want to build or test your software. It's too much overhead.
3. Yes you are right. We really thought we could co-exist with CI runners, and get good performance without reinventing the wheel. But for various reasons that turned out to not be the case. So we're going to ship vertically integrated runners, with excellent scalability and performance. DM me if you want early access :)
TLDR: yes we needed to choose and execute. We have, and we are.
Dagger was something I looked into two or so years ago before they got consumed by the LLM and AI agent hype, and while the promise of being able to run the exact CI workflows locally seemed excellent, it seemed that there's basically no way be a Dagger user without buying into their Dagger Cloud product.
I ended up opting for CUE and GitHub Actions, and I'm glad I did as it made everything much, much simpler.
Can you explain/link to why you can't really use this without their cloud product? I'm not seeing anything at a glance, and this looks useful for a project of mine, but I don't want to be trapped by limitations that I only find out about after putting in weeks of work
Overall I like Dagger conceptually, but I wish they'd start focusing more on API stability and documentation (tbf it's not v1.0). v0.19 broke our Dockerfile builds and I don't feel like figuring out the new syntax atm. Having to commit dev time to the upgrade treadmill to keep CI/CD working was not the dream.
re: the cloud specifically see these GitHub issues:
Basically if you want consistently fast cached builds it's a PITA and/or not possible without the cloud product, depending on how you set things up. We do run it self-hosted though, YMMV.
One thing that I liked about switching from a Docker-based solution like Dagger to Nix is that it relaxed the infrastructure requirements to getting good caching properties.
We used Dagger, and later Nix, mostly to implement various kinds of security scans on our codebases using a mix of open-source tools and clients for proprietary ones that my employer purchases. We've been using Nix for years now, and still haven't set up any of our own binary cache. But we still have mostly-cached builds thanks to the public NixOS binary cache, and we hit that relatively sparingly because we run those jobs on bare metal in self-hosted CI runners. Each scan job typically finishes in less than 15 seconds once the cache is warm, and takes up to 3 minutes when the local cache is cold (in case we build a custom dependency).
Some time in the next quarter or two I'll finish our containerization effort for this so that all the jobs on a runner will share a /nix/store and Nix daemon socket bind-mounted from the host, so we can have relatively safe "multi-tenant" runners where all jobs run under different users in rootless Podman containers while still sharing a global cache for all Nix-provided dependencies. Then we get a bit more isolation and free cleanup for all our jobs but we can still keep our pipelines running fast.
We only have a few thousand codebases, so a few big CI boxes should be fine, but if we ever want to autoscale down, it should be possible to convert such EC2 boxes into Kubernetes nodes, which would be a fun learning project for me. Maybe we could get wider sharing that way and stand up fewer runner VMs.
Somewhere on my backlog is experimenting with Cachix, so we should get per-derivation caching as well, which is finer-grained than Docker's layers.
Hi, I'm the founder of Dagger. It's not true that you can't use Dagger without our cloud offering. At the moment our only commercial product is observability for your Dagger pipelines. It's based on standard otel telemetry emitted by our open source engine. It's completely optional.
If you have questions about Dagger, I encourage you to join our Discord server, we will be happy to answer them!
> If you’ve been active in the Dagger community, this news will come as no surprise. Since we released multi-language support, we have seen a steep decline in usage of our original CUE configuration syntax, and have made it clear that feature parity with newer SDKs would not be a priority.
That is, of course, a self-fulfilling prophecy (or, perhaps, a self-inflicted wound). As soon as Dagger's "multi-language support" came out (actually a bit before), the CUE SDK was rendered abandonware. Development only happened on the new backend, and CUE support was never ported over to the new one.
Dagger founder here. We moved away from CUE because the number one complaint from our early users was having to learn CUE. The number two complaint was bugs in the language that we diligently escalated upstream, but would never get fixed, including crippling memory leaks.
We shipped multi-language support because we had no choice. It was a major engineering effort that we hadn't originally planned for, but it was painfully obvious that remaining a CUE-only platform was suicide.
I think multi-language support is a great feature, and I understand why you had to go for it. While I'm sure some people likely switched away from CUE once they had the chance because they weren't interested in working with a novel and perhaps quirky DSL, I'm also sure some stopped using the CUE SDK just because it was clear to them that it was being abandoned— I know that because I'm one of them. I'm one of the users who stopped using the CUE SDK after multi-language support came out— and it's not because I preferred using one of those other languages. That's all I'm saying.
I understand. We really did try to port the CUE SDK over to the new APIs, but there were impedance mistmatches that made it difficult to do so without major breaking changes - basically we would have needed to design a new SDK from scratch. We asked for opinions on our discord, and it felt like there weren't enough people interested to justify that kind of effort.
For a while there was activity on the #cue channel about a community SDK (that's how we got PHP, Java, Rust, Elixir and dotnet), but it didn't materialize.
It looks like you were in the minority that would have preferred to continue using the original CUE SDK - I'm sorry that we didn't find a way to continue supporting it.
> It looks like you were in the minority that would have preferred to continue using the original CUE SDK - I'm sorry that we didn't find a way to continue supporting it.
I admit that when I went to the CUE documentation (because I was learning Dagger!) and read about the idea of thinking of validation as locating both schema and configuration on an infinite type/value lattice and sort of walking down from schema to concrete configuration, I thought "holy shit, it makes perfect sense". I'd never really thought about unification in configuration languages before, and it's a really cool idea, one of those things that's simple enough to be intuitive but also really powerful. First-class deep merge support is something that I really miss in hacky configuration languages like HCL, for example. So right away I felt "these CUE guys are onto something! let's see how it pans out".
It felt painful for me to abandon that investment, as I'm sure it did for many on your team, too.
While I'm fairly happy with what I've managed to do since moving on from using Dagger, the CI space is still a mess overall, and I think it needs tools like Dagger and more. So while it will likely be some time before I reevaluate Dagger for use at my current job, I do still wish you and your team present and continued success!
When I got started it was much more difficult as you had to do a lot of manual work to get things started, and you really had to believe the promises that CUE offered (which I did...), but nowadays they've made so many steps in the right direction that getting something going is far quicker!
Maybe it’s just me, but these sample workflows don’t look less complicated, just another kind of complex? If you’re already heavily using CUE in your project this lateral complexity shift might make sense, but I don’t see why I would start using it…
Like my guy 'diarrhea' already echoed: using CUE absolutely does not make sense at a small scale; just write your YAML and get on with your day. We were using it to generate dozens upon dozens of GitHub Actions workflows from what was essentially a single source of truth, and because CUE can export to JSON too then that single source of truth could then easily be leveraged to provide other input files to be used elsewhere.
To some extent yes. If all you have is 2 GitHub Actions YAML files you are not going to reap massive benefits.
I'm a big fan of CUE myself. The benefits compound as you need to output more and more artifacts (= YAML config). Think of several k8s manifests, several GitHub Actions files, e.g. for building across several combinations of OSes, settings, etc.
CUE strikes a really nice balance between being primarily data description and not a Turing-complete language (e.g. cdk8s can get arbitrarily complex and abstract), reducing boilerplate (having you spell out the common bits once only, and each non-commit bit once only) and being type-safe (validation at build/export time, with native import of Go types, JSON schema and more).
They recently added an LSP which helps close the gap to other ecosystems. For example, cdk8s being TS means it naturally has fantastic IDE support, which CUE has been lacking in. CUE error messages can also be very verbose and unhelpful.
At work, we generate a couple thousand lines of k8s YAML from ~0.1x of that in CUE. The CUE is commented liberally, and validation imported from native k8s types, and sprinkled in where needed otherwise (e.g. we know for our application the FOO setting needs to be between 5 and 10). The generated YAML is clean, without any indentation and quoting worries. We also generate YAML-in-YAML, i.e. our application takes YAML config, which itself is in an outer k8s YAML ConfigMap. YAML-in-YAML is normally an enormous pain and easy to get wrong. In CUE it's just `yaml.Marshal`.
You get a lot of benefit for a comparatively simple mental model: all your CUE files form just one large document, and for export to YAML it's merged. Any conflicting values and any missing values fail the export. That's it. The mental model of e.g. cdk8s is massively more complex and has unbounded potential for abstraction footguns (being TypeScript). Not to mention CUE is Go and shipped as a single binary; the CUE v0.15.0 you use today will still compile and work 10 years from now.
You can start very simple, with CUE looking not unlike JSON, and add CUE-specific bits from there. You can always rip out the CUE and just keep the generated YAML, or replace CUE with e.g. cdk8s. It's not a one-way door.
The cherry on top are CUE scripts/tasks. In our case we use a CUE script to split the one-large-document (10s of thousands of lines) into separate files, according to some criteria. This is all defined in CUE as well, meaning I can write ~40 lines of CUE (this has a bit of a learning curve) instead of ~200 lines of cursed, buggy bash.
What I don't get is why would someone code in the terrible GitHub actions dsl which only runs on GitHub actions and nowhere else when there are so many other options that run perfectly fine if you just run it from GitHub actions.
I thought Dagger had/has a lot of potential to be "AWS-CDK for CI pipelines".
I.e. declaratively setup a web of CI / deployment tasks, based on docker, with a code-first DSL, instead of the morass of copy-pasted (and yes orbs) CircleCI yaml files we have strewn about our internals repos.
But their DSL for defining your pipelines is ... golang? Like who would pick golang as "a friendly language for setting up configs".
The underlying tech is technically language-agnostic, just as aws-cdk's is (you can share cdk constructs across TypeScript/Python), but it's rooted in golang as the originating/first-class language, so imo will never hit aws-cdk levels of ergonomics.
That technical nit aside, I love the idea; ran a few examples of it a year or so ago and was really impressed with the speed; just couldn't wrap my around "how can I make this look like cdk".
Still looks like "a circa-2000s Java builder API" and doesn't look like pleasant / declarative / idiomatic TypeScript, which is what aws-cdk pulled off.
Genuinely impressively (imo), aws-cdk intermixes "it's declarative" (you're setting up your desired state) but also "it's code" (you can use all the usual abstractions) in a way that is pretty great & unique.
Could you share an example of aws-cdk code that you think Dagger should take inspiration from? Dagger and aws-cdk work very differently under the hood, so it's difficult to make an apples-to-apples comparison. If there's a way to make Dagger more TS-native without sacrificing other important properties of Dagger, I'm interested. Thanks.
Hi, I'm the founder of Dagger. I can't speak to the negativity, but if you're looking for a way to make your CI more portable, I recommend joining our Discord and asking our community directly about the pros and cons of using Dagger. Even if you don't end up using it, there are a lot of people there who are passionate about CI and can recommend other alternatives, in a more constructive and pragmatic way than you are getting here.
I loved the original promise of Dagger and it’s still 90% great.
But one flaw (IMO) that it can’t export artifacts and import into other steps without breaking the cache.
Eg if you provide monorepo as input, and then on some step narrow your build to one specific dir, then even when files change outside of that dir then caching still is invalidated.
Which makes it extremely verbose and maintenance nightmare to keep multiple narrow inputs and keep all those paths up to date.
You can filter input directories before they are loaded, to avoid this. There's no reason you shouldn't be able to get precise cache invalidation in a large monorepo. If you want, DM me on the Dagger Discord and I'll help you out!
I used the old CUE-based version when it came out, and was really excited about it. I liked it, and enjoyed working with CUE, but the API was clunky and incomplete.
Then they completely abandoned not just the CUE frontend, but CUE altogether (while strenuously denying that they were doing so) for a GraphQL-based rewrite that focused on letting people use popular general-purpose languages to construct their workflows. The initial rollout of this was not feature complete and only supported imperative languages (Python and TypeScript, IIRC), which I didn't like.
Instead of porting everything over to all their new interfaces, I hopped off the train and rewrote all of our portable pipeline scripts in Nix, via Devenv. At the time, I'd never used Devenv before, but getting the work done that time still took maybe a tenth of the time or less. More than anything else, this was due to not having to fuck around with the additional overhead Docker entails (fussing with mount points, passing files from one stage to another, rebuilding images, setting up VMs... all of it). I got the reproducibility without the extra bullshit, and got to work with interfaces that have proven much more stable.
I still think there's a place for something like Dagger, focused just on CI, perhaps even still using Docker as a deployment/distribution strategy. But I no longer trust Dagger to execute on that. I think a proper external DSL (probably special-purposw but still Turing-complete, e.g., Nickel) is the right fit for this domain, and that it should support multiple means of achieving repeatability rather than just Docker (e.g., Nix on bare metal and Podman, to start). An option to work on bare metal via reproducible environment management tools like Nix, Guix, or Spack is a valuable alternative to burdensome approaches like containers.
I haven't looked at Dagger in several months, but the other big piece that is missing for portable CI workflows is a library that abstracts over popular CI platforms so you can easily configure pull/merge request pipelines without worrying about the implementation details like what environment variables each platform exposes to indicate source and target branch.
Idk anything about all the AI horseshit; I was off the Dagger bandwagon before they took that turn. I don't know if it's serious or a nominal play to court investors. But that kind of pivot is another reason not to build core infra on top of the work of startups imo. If the product is 70% of what you want, you have no way of knowing whether filling that 30% gap is something the maintainers will suddenly pivot away from, even if their current direction looks aligned with yours.
I'd recommend considering tools in this space only if (a) they're already close to 100% of what you need and (b) they're open-source. Maybe you can relax (a) if it's really easy to extend the codebase (I find this to be true for Devenv's Nix modules, for example.)
Do you have any examples of your Devenv workflow you can share? I took a look at Dagger and really like the concept, but I'm trying to figure out the limitations/why there's so much negativity in this thread.
I currently manage my development environments via NixOS and Devenv, so if I could just keep that and achieve the same functionality, that sounds good to me.
At the time, Devenv was still pretty barebones compared to now (the rate of feature development has been great in the past few years), so it didn't have much in the way of support for workflows per se.
Instead I used the scripts module to write shell scripts that bring along all of their dependencies. Thus all of our GitLab jobs are just one-line invocations of our wrapper scripts. We currently use a flake.nix-based Devenv and expose the scripts from the Devenv module as flake outputs, then just run them with `nix run`.
Nowadays, though, Devenv actually directly supports some relevant functionality in the form of tasks¹, which can be written in any language, and support dependency relationships and run concurrently by default.
We use those in a few places as well, though mostly in place of older, cruder `enterShell` integration. There, we use them:
- to seamlessly ensure correct state of Git extensions:
- all Git submodules are checked out
- Git-LFS is configured and all large files are correctly checked out
- to ensure valid OpenTofu caches since we use Nix instead of OpenTofu for managing providers
- if .terraform dir doesn't exist, run `tofu init`
- if Terraform providers lock file is older than devenv lock, run `tofu init -upgrade`
- manage Poetry stubs and virtualenvs (most of this is configured behind the scenes upstream, but we do some extra stuff)
- manage Bundler dependencies and binstubs for Ruby projects so executables under development can be involed naively from PATH (much of this is bespoke but upstreamable)
So a lot of those Devenv tasks are things you might do in build or pre-build stages of a CI pipeline. You can define the requisite dependency relationships and then just run `devenv test` or `devenv tasks ...` as your CI job.
What we don't have is any abstraction over the various CI platforms we use (currently GitLab, GitHub, and AzDO, with Jenkins support in the backlog). Unfortunately, for the sake of a better DX for our (internal) customers, we still integrate fairly directly with each CI platform, so we do have piles of vendor-specific YAML.
Our tasks and scripts are currently just Bash, but they're relatively nice Bash because their dependencies are managed with resholve and they're linted by ShellCheck, thanks to some experimental changes I made to Devenv's scripts module.
For our security/quality scans, we do, however, have some logic nestled in those Bash scripts which checks for a CI environment via env vars, then performs differential rather than full scans if we're in merge request contexts. Where the upstream tools don't directly support this, we basically perform "before" and "after" scans and take a diff. This is the direction I'd like to go in the future— all in on this, surfacing jobs in a "native" way be damned.
Unfortunately I can't directly share these examples as they belong to my employer and live in proprietary projects. But what I can do is open a couple of WIP PRs for the upstreamable work (new or modified Devenv modules) so you can take a look at that at least.
Also fwiw I think Dagger is still worth checking out. I just feel a bit burned by adopting it too early in its lifecycle, and generally advice caution about adopting tools developed by VC-backed startups.
People likewise won't confuse a DI framework for Android with a GitHub Actions replacement. Complaints about overlapping names would make sense if both Daggers were DI frameworks or Github Actions replacements, but they're not.
I was interested in the beginning for CI/CD, but then they tried to take a kind of "AI-oriented" view in order to ride the AI wave, and the value prop of their tool was completely muddied up...
I've tried it, but there's too much "sorry not in the open-source edition, please buy the entreprise edition" stuff all around, which makes it quite unusable
Works fine for us for glueing a bunch of CI steps together which would've been a pile of bash otherwise. Works well with depot.dev caches. We don't use it for anything AI either.
I'd say it's somewhere in between. Sure it's marketed in the CI space, but to me the selling point of Dagger is not so much "write your GitHub workflows/GitLab CI in JavaScript" but "local exec, sandboxing, determinism, and fine-grained (remote) caching for mere mortals". So comparing it to Bazel/Buck2 is reasonable.
I wouldn't because build systems like Bazel are declarative and dagger is imperative. I accidentally created a build system in dagger and saw the difference; the code based way was highly branched, and thus unwieldy. I think you would want to call bazel from dagger to handle the build step.
But that dependency order is usually just one big blob of ”COPY src/ . + RUN make”, within that block you have none of the benefits. Bazel/Buck has much finer awareness down to every individual file.
Out of curiosity, would it be feasible to take a big cmake project and generate thousands of compile rules into dagger and use it as a substitute for make with sandboxing? I’ve never seen builkit used with such many nodes, how would it fare?
I use it to do builds in our monorepo. We got onboard before the LLM trash features. The base design is ok but there's things I'd do different today if I knew the build stuff would fade away for the LLM push.
I might be getting old but the videos are too fast for me to understand. Why can't they just put the full command text and the output of it instead of a video.
This looks interesting but I’m trying to understand it in more layman’s terms. Is it more about providing abstractions for llms to work within to do things?
I've never tried it. My first impression based purely on reading the homepage is it adds complexity to something I can already do with a Dockerfile and bash. What can it do that I can't already do more simply?
It does a pretty good job of caching and that does help speed up builds. I also run all of my end to end tests from it because I can coordinate secrets and clusters of containers through it.
If I understand correctly, this is essentially a more composable way to write Dockerfiles? That alone is a very welcome improvement. They would do themselves a big favor if they were more clear on that in their marketing, instead of boasting around the bush with all kinds of other terminology and claims of redefining foundations.
If I already have a Dockerfile that doesn’t need composition, how does this help me vs being a small cosmetic improvement over ”docker build” command line?
A lot of the comments here feel like they're disappointed that this is a "Docker with unnecessary LLM crap thrown in" when I think what they're really going for is more "LLM workflows with a higher degree of observability and sanity".
I think a more interesting point of comparison is the Claude Code Github Action, Co-Pilot code reviews, etc.
I think it started as some kind of CI/CD tools, then they jumped on the AI hype and they started to use it to make it possible to run agents in containers easily... perhaps to do automated actions on CI/CD pipelines that use agents (eg try to solve some minor bugs automatically when you push on a branch, etc)
Although I'm not sure if that's so much a value-added? It's not so hard to just create a container and launch an agent in it.
The whole interesting thing was to use actual programming languages for Docker build, which I think was what they initially tried to do, but now it's a bit incomprehensible... I guess conceptually Dagger relates to Dockerfile a bit like Pulumi related to Terraform?
As someone that has used Dagger a lot (a previous daggernaut / ambassador dropped off after LLMs was announced, and was changing jobs at the time. implemented it at a previous company across 95% of services, built the rust sdk) the approach was and is amazing for building complex build chains.
It serves a place where a dockerfile is not enough, and CI workflows are too difficult to debug or reason about.
I do have some current problems with it though:
1. I don't care at all about the LLM agent workflows, I get that it is possible, but the same people that chose dagger for what it was, is not the same audience that runs agents like that. I can't choose dagger currently, because I don't know if they align with my interests as an engineer solving a specific problems for where I work (delivering software, not running agents).
2. I advocated for modules before it was a thing, but I never implemented it. It is too much magic, I want to write code, not a DSL that looks like code, dagger is already special in that regard, to modules takes it a step too far. You can't find the code in their docs anymore, but dagger can be written with just a .go, .py or .rs file. Simply take in dagger as a dependency and build your workflow.
3. Too complex to operate, dagger doesn't have runners currently, and it is difficult to run a setup in production for CI yourself, without running it in the actions themselves, which can be disastrous for build times, as dagger often leads you into using quite a few images, so having a cache is a must.
Dagger needs to choose and execute; not having runners, even when we we're willing to throw money at them was a mistake IMO. Love the tool, the team, the vision but it is too distracted, magical and impatient to pick up at the moment.
Hi Kasper, good to see you here! Thank you for the detailed feedback.
1. Yes we got over-excited with the agent runtime use case. We stand by the LLM implementation because we never compromised on the integrity of Dagger's modular design. But our marketing and product priorities were all over the place. We're going to refocus on the original use case: helping you ship software, and more particularly building & testing it.
2. Modules have warts but they are essential. We will continue to improve them, and remain committed to them. Without this feature, you have to write a complete standalone program every time you want to build or test your software. It's too much overhead.
3. Yes you are right. We really thought we could co-exist with CI runners, and get good performance without reinventing the wheel. But for various reasons that turned out to not be the case. So we're going to ship vertically integrated runners, with excellent scalability and performance. DM me if you want early access :)
TLDR: yes we needed to choose and execute. We have, and we are.
Thank you again for the feedback.
Sounds good Solomon I look forward to seeing how it goes in about a year when I am going to tackle our CI again ;)
Best of luck and thx for taking my harsh feedback in strides!
Dagger was something I looked into two or so years ago before they got consumed by the LLM and AI agent hype, and while the promise of being able to run the exact CI workflows locally seemed excellent, it seemed that there's basically no way be a Dagger user without buying into their Dagger Cloud product.
I ended up opting for CUE and GitHub Actions, and I'm glad I did as it made everything much, much simpler.
Same - we began the migration to Dagger but then switched to just Docker-In-Docker and custom scripts which run vendor-agnostic
Can you explain/link to why you can't really use this without their cloud product? I'm not seeing anything at a glance, and this looks useful for a project of mine, but I don't want to be trapped by limitations that I only find out about after putting in weeks of work
Overall I like Dagger conceptually, but I wish they'd start focusing more on API stability and documentation (tbf it's not v1.0). v0.19 broke our Dockerfile builds and I don't feel like figuring out the new syntax atm. Having to commit dev time to the upgrade treadmill to keep CI/CD working was not the dream.
re: the cloud specifically see these GitHub issues:
https://github.com/dagger/dagger/issues/6486
https://github.com/dagger/dagger/issues/8004
Basically if you want consistently fast cached builds it's a PITA and/or not possible without the cloud product, depending on how you set things up. We do run it self-hosted though, YMMV.
One thing that I liked about switching from a Docker-based solution like Dagger to Nix is that it relaxed the infrastructure requirements to getting good caching properties.
We used Dagger, and later Nix, mostly to implement various kinds of security scans on our codebases using a mix of open-source tools and clients for proprietary ones that my employer purchases. We've been using Nix for years now, and still haven't set up any of our own binary cache. But we still have mostly-cached builds thanks to the public NixOS binary cache, and we hit that relatively sparingly because we run those jobs on bare metal in self-hosted CI runners. Each scan job typically finishes in less than 15 seconds once the cache is warm, and takes up to 3 minutes when the local cache is cold (in case we build a custom dependency).
Some time in the next quarter or two I'll finish our containerization effort for this so that all the jobs on a runner will share a /nix/store and Nix daemon socket bind-mounted from the host, so we can have relatively safe "multi-tenant" runners where all jobs run under different users in rootless Podman containers while still sharing a global cache for all Nix-provided dependencies. Then we get a bit more isolation and free cleanup for all our jobs but we can still keep our pipelines running fast.
We only have a few thousand codebases, so a few big CI boxes should be fine, but if we ever want to autoscale down, it should be possible to convert such EC2 boxes into Kubernetes nodes, which would be a fun learning project for me. Maybe we could get wider sharing that way and stand up fewer runner VMs.
Somewhere on my backlog is experimenting with Cachix, so we should get per-derivation caching as well, which is finer-grained than Docker's layers.
Hi, I'm the founder of Dagger. It's not true that you can't use Dagger without our cloud offering. At the moment our only commercial product is observability for your Dagger pipelines. It's based on standard otel telemetry emitted by our open source engine. It's completely optional.
If you have questions about Dagger, I encourage you to join our Discord server, we will be happy to answer them!
dagger was originally CUE-based, but there was not enough demand so it was dropped. https://dagger.io/blog/ending-cue-support
> If you’ve been active in the Dagger community, this news will come as no surprise. Since we released multi-language support, we have seen a steep decline in usage of our original CUE configuration syntax, and have made it clear that feature parity with newer SDKs would not be a priority.
That is, of course, a self-fulfilling prophecy (or, perhaps, a self-inflicted wound). As soon as Dagger's "multi-language support" came out (actually a bit before), the CUE SDK was rendered abandonware. Development only happened on the new backend, and CUE support was never ported over to the new one.
Dagger founder here. We moved away from CUE because the number one complaint from our early users was having to learn CUE. The number two complaint was bugs in the language that we diligently escalated upstream, but would never get fixed, including crippling memory leaks.
We shipped multi-language support because we had no choice. It was a major engineering effort that we hadn't originally planned for, but it was painfully obvious that remaining a CUE-only platform was suicide.
Thanks for the response.
I think multi-language support is a great feature, and I understand why you had to go for it. While I'm sure some people likely switched away from CUE once they had the chance because they weren't interested in working with a novel and perhaps quirky DSL, I'm also sure some stopped using the CUE SDK just because it was clear to them that it was being abandoned— I know that because I'm one of them. I'm one of the users who stopped using the CUE SDK after multi-language support came out— and it's not because I preferred using one of those other languages. That's all I'm saying.
I understand. We really did try to port the CUE SDK over to the new APIs, but there were impedance mistmatches that made it difficult to do so without major breaking changes - basically we would have needed to design a new SDK from scratch. We asked for opinions on our discord, and it felt like there weren't enough people interested to justify that kind of effort.
For a while there was activity on the #cue channel about a community SDK (that's how we got PHP, Java, Rust, Elixir and dotnet), but it didn't materialize.
It looks like you were in the minority that would have preferred to continue using the original CUE SDK - I'm sorry that we didn't find a way to continue supporting it.
> It looks like you were in the minority that would have preferred to continue using the original CUE SDK - I'm sorry that we didn't find a way to continue supporting it.
I admit that when I went to the CUE documentation (because I was learning Dagger!) and read about the idea of thinking of validation as locating both schema and configuration on an infinite type/value lattice and sort of walking down from schema to concrete configuration, I thought "holy shit, it makes perfect sense". I'd never really thought about unification in configuration languages before, and it's a really cool idea, one of those things that's simple enough to be intuitive but also really powerful. First-class deep merge support is something that I really miss in hacky configuration languages like HCL, for example. So right away I felt "these CUE guys are onto something! let's see how it pans out".
It felt painful for me to abandon that investment, as I'm sure it did for many on your team, too.
While I'm fairly happy with what I've managed to do since moving on from using Dagger, the CI space is still a mess overall, and I think it needs tools like Dagger and more. So while it will likely be some time before I reevaluate Dagger for use at my current job, I do still wish you and your team present and continued success!
Thanks again for dropping by and listening.
Do you have examples of your CUE and Github Actions setup?
I see someone else asked below:
https://news.ycombinator.com/item?id=46262846
Same: the promise of defining CI/CD in code is good, but the implementation didn't make sense to me even before the LLM stuff
Do you have any more details on using Cue with GHA? I've also looked at Dagger and been quite disappointed with it (and their terrible documentation).
When I got started it was much more difficult as you had to do a lot of manual work to get things started, and you really had to believe the promises that CUE offered (which I did...), but nowadays they've made so many steps in the right direction that getting something going is far quicker!
Here are a few links to whet your appetite:
- https://cue.dev/docs/getting-started-with-github-actions-cue...
- https://cue.dev/docs/drying-up-github-actions-workflows/
- https://cue.dev/docs/spotting-errors-earlier-github-actions-...
Definitely read through the CUE documentation (https://cuelang.org/docs/), watch their YouTube videos (https://www.youtube.com/@cuelang/videos), and join the community Slack channel (https://cuelang.org/community/). I've gotten a lot of help in the Slack from both enthusiastic community members and from the developers themselves whenever I've gotten stuck.
Maybe it’s just me, but these sample workflows don’t look less complicated, just another kind of complex? If you’re already heavily using CUE in your project this lateral complexity shift might make sense, but I don’t see why I would start using it…
Like my guy 'diarrhea' already echoed: using CUE absolutely does not make sense at a small scale; just write your YAML and get on with your day. We were using it to generate dozens upon dozens of GitHub Actions workflows from what was essentially a single source of truth, and because CUE can export to JSON too then that single source of truth could then easily be leveraged to provide other input files to be used elsewhere.
> just another kind of complex?
To some extent yes. If all you have is 2 GitHub Actions YAML files you are not going to reap massive benefits.
I'm a big fan of CUE myself. The benefits compound as you need to output more and more artifacts (= YAML config). Think of several k8s manifests, several GitHub Actions files, e.g. for building across several combinations of OSes, settings, etc.
CUE strikes a really nice balance between being primarily data description and not a Turing-complete language (e.g. cdk8s can get arbitrarily complex and abstract), reducing boilerplate (having you spell out the common bits once only, and each non-commit bit once only) and being type-safe (validation at build/export time, with native import of Go types, JSON schema and more).
They recently added an LSP which helps close the gap to other ecosystems. For example, cdk8s being TS means it naturally has fantastic IDE support, which CUE has been lacking in. CUE error messages can also be very verbose and unhelpful.
At work, we generate a couple thousand lines of k8s YAML from ~0.1x of that in CUE. The CUE is commented liberally, and validation imported from native k8s types, and sprinkled in where needed otherwise (e.g. we know for our application the FOO setting needs to be between 5 and 10). The generated YAML is clean, without any indentation and quoting worries. We also generate YAML-in-YAML, i.e. our application takes YAML config, which itself is in an outer k8s YAML ConfigMap. YAML-in-YAML is normally an enormous pain and easy to get wrong. In CUE it's just `yaml.Marshal`.
You get a lot of benefit for a comparatively simple mental model: all your CUE files form just one large document, and for export to YAML it's merged. Any conflicting values and any missing values fail the export. That's it. The mental model of e.g. cdk8s is massively more complex and has unbounded potential for abstraction footguns (being TypeScript). Not to mention CUE is Go and shipped as a single binary; the CUE v0.15.0 you use today will still compile and work 10 years from now.
You can start very simple, with CUE looking not unlike JSON, and add CUE-specific bits from there. You can always rip out the CUE and just keep the generated YAML, or replace CUE with e.g. cdk8s. It's not a one-way door.
The cherry on top are CUE scripts/tasks. In our case we use a CUE script to split the one-large-document (10s of thousands of lines) into separate files, according to some criteria. This is all defined in CUE as well, meaning I can write ~40 lines of CUE (this has a bit of a learning curve) instead of ~200 lines of cursed, buggy bash.
Interesting approach! I can see promise in the vet tooling assuming the module keeps up with GitHub's schema well.
If you had to pick, what's the "killer" feature for you with this module?
What I don't get is why would someone code in the terrible GitHub actions dsl which only runs on GitHub actions and nowhere else when there are so many other options that run perfectly fine if you just run it from GitHub actions.
I thought Dagger had/has a lot of potential to be "AWS-CDK for CI pipelines".
I.e. declaratively setup a web of CI / deployment tasks, based on docker, with a code-first DSL, instead of the morass of copy-pasted (and yes orbs) CircleCI yaml files we have strewn about our internals repos.
But their DSL for defining your pipelines is ... golang? Like who would pick golang as "a friendly language for setting up configs".
The underlying tech is technically language-agnostic, just as aws-cdk's is (you can share cdk constructs across TypeScript/Python), but it's rooted in golang as the originating/first-class language, so imo will never hit aws-cdk levels of ergonomics.
That technical nit aside, I love the idea; ran a few examples of it a year or so ago and was really impressed with the speed; just couldn't wrap my around "how can I make this look like cdk".
They have SDKs in many languages, not just Go. I use the python one. And they use code, not a DSL.
Right, my point is that this:
https://docs.dagger.io/cookbook/services?sdk=typescript
Still looks like "a circa-2000s Java builder API" and doesn't look like pleasant / declarative / idiomatic TypeScript, which is what aws-cdk pulled off.
Genuinely impressively (imo), aws-cdk intermixes "it's declarative" (you're setting up your desired state) but also "it's code" (you can use all the usual abstractions) in a way that is pretty great & unique.
Could you share an example of aws-cdk code that you think Dagger should take inspiration from? Dagger and aws-cdk work very differently under the hood, so it's difficult to make an apples-to-apples comparison. If there's a way to make Dagger more TS-native without sacrificing other important properties of Dagger, I'm interested. Thanks.
And for those who may not know the founder of Dagger is the same guy who founded Docker - Solomon Hykes.
Wow, the comments are generally negative. That’s sad. I was about to create a POC to present to my team.
What else could be used to abstract away your CICD from the launcher (Jenkins, Argo Workflows, GitHub Actions, etc.)?
Hi, I'm the founder of Dagger. I can't speak to the negativity, but if you're looking for a way to make your CI more portable, I recommend joining our Discord and asking our community directly about the pros and cons of using Dagger. Even if you don't end up using it, there are a lot of people there who are passionate about CI and can recommend other alternatives, in a more constructive and pragmatic way than you are getting here.
I loved the original promise of Dagger and it’s still 90% great.
But one flaw (IMO) that it can’t export artifacts and import into other steps without breaking the cache.
Eg if you provide monorepo as input, and then on some step narrow your build to one specific dir, then even when files change outside of that dir then caching still is invalidated.
Which makes it extremely verbose and maintenance nightmare to keep multiple narrow inputs and keep all those paths up to date.
You can filter input directories before they are loaded, to avoid this. There's no reason you shouldn't be able to get precise cache invalidation in a large monorepo. If you want, DM me on the Dagger Discord and I'll help you out!
Anybody using it extensively? It doesn't seem to have made the splash I expected it to at launch
I used the old CUE-based version when it came out, and was really excited about it. I liked it, and enjoyed working with CUE, but the API was clunky and incomplete.
Then they completely abandoned not just the CUE frontend, but CUE altogether (while strenuously denying that they were doing so) for a GraphQL-based rewrite that focused on letting people use popular general-purpose languages to construct their workflows. The initial rollout of this was not feature complete and only supported imperative languages (Python and TypeScript, IIRC), which I didn't like.
Instead of porting everything over to all their new interfaces, I hopped off the train and rewrote all of our portable pipeline scripts in Nix, via Devenv. At the time, I'd never used Devenv before, but getting the work done that time still took maybe a tenth of the time or less. More than anything else, this was due to not having to fuck around with the additional overhead Docker entails (fussing with mount points, passing files from one stage to another, rebuilding images, setting up VMs... all of it). I got the reproducibility without the extra bullshit, and got to work with interfaces that have proven much more stable.
I still think there's a place for something like Dagger, focused just on CI, perhaps even still using Docker as a deployment/distribution strategy. But I no longer trust Dagger to execute on that. I think a proper external DSL (probably special-purposw but still Turing-complete, e.g., Nickel) is the right fit for this domain, and that it should support multiple means of achieving repeatability rather than just Docker (e.g., Nix on bare metal and Podman, to start). An option to work on bare metal via reproducible environment management tools like Nix, Guix, or Spack is a valuable alternative to burdensome approaches like containers.
I haven't looked at Dagger in several months, but the other big piece that is missing for portable CI workflows is a library that abstracts over popular CI platforms so you can easily configure pull/merge request pipelines without worrying about the implementation details like what environment variables each platform exposes to indicate source and target branch.
Idk anything about all the AI horseshit; I was off the Dagger bandwagon before they took that turn. I don't know if it's serious or a nominal play to court investors. But that kind of pivot is another reason not to build core infra on top of the work of startups imo. If the product is 70% of what you want, you have no way of knowing whether filling that 30% gap is something the maintainers will suddenly pivot away from, even if their current direction looks aligned with yours.
I'd recommend considering tools in this space only if (a) they're already close to 100% of what you need and (b) they're open-source. Maybe you can relax (a) if it's really easy to extend the codebase (I find this to be true for Devenv's Nix modules, for example.)
Do you have any examples of your Devenv workflow you can share? I took a look at Dagger and really like the concept, but I'm trying to figure out the limitations/why there's so much negativity in this thread.
I currently manage my development environments via NixOS and Devenv, so if I could just keep that and achieve the same functionality, that sounds good to me.
At the time, Devenv was still pretty barebones compared to now (the rate of feature development has been great in the past few years), so it didn't have much in the way of support for workflows per se.
Instead I used the scripts module to write shell scripts that bring along all of their dependencies. Thus all of our GitLab jobs are just one-line invocations of our wrapper scripts. We currently use a flake.nix-based Devenv and expose the scripts from the Devenv module as flake outputs, then just run them with `nix run`.
Nowadays, though, Devenv actually directly supports some relevant functionality in the form of tasks¹, which can be written in any language, and support dependency relationships and run concurrently by default.
We use those in a few places as well, though mostly in place of older, cruder `enterShell` integration. There, we use them: - to seamlessly ensure correct state of Git extensions: - all Git submodules are checked out - Git-LFS is configured and all large files are correctly checked out - to ensure valid OpenTofu caches since we use Nix instead of OpenTofu for managing providers - if .terraform dir doesn't exist, run `tofu init` - if Terraform providers lock file is older than devenv lock, run `tofu init -upgrade` - manage Poetry stubs and virtualenvs (most of this is configured behind the scenes upstream, but we do some extra stuff) - manage Bundler dependencies and binstubs for Ruby projects so executables under development can be involed naively from PATH (much of this is bespoke but upstreamable)
So a lot of those Devenv tasks are things you might do in build or pre-build stages of a CI pipeline. You can define the requisite dependency relationships and then just run `devenv test` or `devenv tasks ...` as your CI job.
What we don't have is any abstraction over the various CI platforms we use (currently GitLab, GitHub, and AzDO, with Jenkins support in the backlog). Unfortunately, for the sake of a better DX for our (internal) customers, we still integrate fairly directly with each CI platform, so we do have piles of vendor-specific YAML.
Our tasks and scripts are currently just Bash, but they're relatively nice Bash because their dependencies are managed with resholve and they're linted by ShellCheck, thanks to some experimental changes I made to Devenv's scripts module.
For our security/quality scans, we do, however, have some logic nestled in those Bash scripts which checks for a CI environment via env vars, then performs differential rather than full scans if we're in merge request contexts. Where the upstream tools don't directly support this, we basically perform "before" and "after" scans and take a diff. This is the direction I'd like to go in the future— all in on this, surfacing jobs in a "native" way be damned.
Unfortunately I can't directly share these examples as they belong to my employer and live in proprietary projects. But what I can do is open a couple of WIP PRs for the upstreamable work (new or modified Devenv modules) so you can take a look at that at least.
Also fwiw I think Dagger is still worth checking out. I just feel a bit burned by adopting it too early in its lifecycle, and generally advice caution about adopting tools developed by VC-backed startups.
--
1: https://devenv.sh/tasks/
This seemed cool as it looked like a new CI/CD tool or IaC system.
Then... it wasn't. The more I read the less I ever want to see this again. The LLM train has got to end at some point.
Dagger is already a very popular thing -- a DI framework
It's also a type of blade, been around for hundreds of years!
/s I've never heard of Dagger the DI framework but I have heard of this Dagger. Names will overlap sometimes and it's not a big deal.
People aren’t going to confuse a physical knife with this.
Android is the most popular operating system on earth. Names can overlap but you shouldn’t choose one where you can’t live up to owning it.
People likewise won't confuse a DI framework for Android with a GitHub Actions replacement. Complaints about overlapping names would make sense if both Daggers were DI frameworks or Github Actions replacements, but they're not.
> can’t live up to owning it.
?
Yeah I initially thought of this coming from android
The title says "dev environments". Can dagger be used for creatin dev environments, like devcontainers or https://www.jetify.com/docs/devbox/quickstart ?
I was interested in the beginning for CI/CD, but then they tried to take a kind of "AI-oriented" view in order to ride the AI wave, and the value prop of their tool was completely muddied up...
Same, the marketing was even worse a few months ago.
Curious if anyone in the thread has / is using windmill?
They don't seem to have jumped for AI hype (yet?)...
https://www.windmill.dev/
have used it, and i do like it, but the licensing situation is not great. It open source but its not free software by any means.
I've tried it, but there's too much "sorry not in the open-source edition, please buy the entreprise edition" stuff all around, which makes it quite unusable
Anybody used it?
Without the LLM bits, this is basically like Bazel or buck2, right?
Works fine for us for glueing a bunch of CI steps together which would've been a pile of bash otherwise. Works well with depot.dev caches. We don't use it for anything AI either.
Founder of Depot here! Glad to hear it’s working for you all. Always happy to help expand things or make things better if you ever have ideas.
No, it's code based CI/CD not a build system.
I'd say it's somewhere in between. Sure it's marketed in the CI space, but to me the selling point of Dagger is not so much "write your GitHub workflows/GitLab CI in JavaScript" but "local exec, sandboxing, determinism, and fine-grained (remote) caching for mere mortals". So comparing it to Bazel/Buck2 is reasonable.
I wouldn't because build systems like Bazel are declarative and dagger is imperative. I accidentally created a build system in dagger and saw the difference; the code based way was highly branched, and thus unwieldy. I think you would want to call bazel from dagger to handle the build step.
Dagger uses BuildKit. all your “imperative” code does it assemble a graph that’s gonna be executed by BuildKit in dependency order.
But that dependency order is usually just one big blob of ”COPY src/ . + RUN make”, within that block you have none of the benefits. Bazel/Buck has much finer awareness down to every individual file.
Out of curiosity, would it be feasible to take a big cmake project and generate thousands of compile rules into dagger and use it as a substitute for make with sandboxing? I’ve never seen builkit used with such many nodes, how would it fare?
Dagger is a declarative DAG engine. So, yes, you can do that.
There is an overhead per container launched so it would probably not be worth it.
Yes, but that does not help you build declaratively in dagger, does it? I would like to see an example of how to do so, if you have one.
Its not as fine grained as bazel/buck. That doesnt necessarily matter.
Yeah, it's more like a .NET Aspire
I use it to do builds in our monorepo. We got onboard before the LLM trash features. The base design is ok but there's things I'd do different today if I knew the build stuff would fade away for the LLM push.
I might be getting old but the videos are too fast for me to understand. Why can't they just put the full command text and the output of it instead of a video.
This looks interesting but I’m trying to understand it in more layman’s terms. Is it more about providing abstractions for llms to work within to do things?
It’s actually a CI/CD as code tool, where some pieces can be LLM agents.
But, the marketing heavily focuses on LLM stuff to the point of making everyone confused.
I've never tried it. My first impression based purely on reading the homepage is it adds complexity to something I can already do with a Dockerfile and bash. What can it do that I can't already do more simply?
It does a pretty good job of caching and that does help speed up builds. I also run all of my end to end tests from it because I can coordinate secrets and clusters of containers through it.
Instead of YAML workflows you write code, and there is an analog of the Github Actions Marketplace (the Daggerverse).
If I understand correctly, this is essentially a more composable way to write Dockerfiles? That alone is a very welcome improvement. They would do themselves a big favor if they were more clear on that in their marketing, instead of boasting around the bush with all kinds of other terminology and claims of redefining foundations.
If I already have a Dockerfile that doesn’t need composition, how does this help me vs being a small cosmetic improvement over ”docker build” command line?
A lot of the comments here feel like they're disappointed that this is a "Docker with unnecessary LLM crap thrown in" when I think what they're really going for is more "LLM workflows with a higher degree of observability and sanity".
I think a more interesting point of comparison is the Claude Code Github Action, Co-Pilot code reviews, etc.
why do i need this when we got docker? anyone mind explaining?
I think it started as some kind of CI/CD tools, then they jumped on the AI hype and they started to use it to make it possible to run agents in containers easily... perhaps to do automated actions on CI/CD pipelines that use agents (eg try to solve some minor bugs automatically when you push on a branch, etc)
Although I'm not sure if that's so much a value-added? It's not so hard to just create a container and launch an agent in it.
The whole interesting thing was to use actual programming languages for Docker build, which I think was what they initially tried to do, but now it's a bit incomprehensible... I guess conceptually Dagger relates to Dockerfile a bit like Pulumi related to Terraform?