This seems like the kind of technology that could make the problem described in https://www.gnu.org/philosophy/can-you-trust.en.html a lot worse. Do you have any plans for making sure it doesn't get used for that?
I'm Aleksa, one of the founding engineers. We will share more about this in the coming months but this is not the direction nor intention of what we are working on. The models we have in mind for attestation are very much based on users having full control of their keys. This is not just a matter of user freedom, in practice being able to do this is far more preferable for enterprises with strict security controls.
I've been a FOSS guy my entire adult life, I wouldn't put my name to something that would enable the kinds of issues you describe.
Thanks, this would be helpful. I will follow on by recommending that you always make it a point to note how user freedom will be preserved, without using obfuscating corpo-speak or assuming that users don’t know what they want, when planning or releasing products. If you can maintain this approach then you should be able to maintain a good working relationship with the community. If you fight the community you will burn a lot of goodwill and will have to spend resources on PR. And there is only so much that PR can do!
Better security is good in theory, as long as the user maintains control and the security is on the user end. The last thing we need is required ID linked attestation for accessing websites or something similar.
Thanks for the reassurance, the first ray of sunshine in this otherwise rather alarming thread. Your words ring true.
It would be a lot more reassuring if we knew what the business model actually was, or indeed anything else at all about this. I remain somewhat confused as to the purpose of this announcement when no actual information seems to be forthcoming. The negative reactions seen here were quite predictable, given the sensitive topic and the little information we do have.
"We are confident we have a very robust path to revenue."
I take it that you are not at this stage able to provide details of the nature of the path to revenue. On what kind of timescale do you envisage being able to disclose your revenue stream/subscribers/investors?
As I understand it, the main customers for this sort of thing are companies making Tivo-style products - where they want to use Linux in their product, but they want to lock it down so it can't be modified by the device owner.
This can be pretty profitable; once your customers have rolled out a fleet of hardware locked down to only run kernels you've signed.
I just want more trustworthy systems. This particular concept of combining reproducible builds, remote attestation and transparency logs is something I came up with in 2018. My colleagues and I started working on it, took a detour into hardware (tillitis.se) and kind of got stuck on the transparency part (sigsum.org, transparency.dev, witness-network.org).
Then we discovered snapshot.debian.org wasn't feeling well, so that was another (important) detour.
Part of me wish we had focused more on getting System Transparency in its entirety in production at Mullvad. On the other hand I certainly don't regret us creating Tillitis TKey, Sigsum, taking care of Debian Snapshot service, and several other things.
Now, six years later, systemd and other projects have gotten a long way to building several of the things we need for ST. It doesn't make sense to do double work, so I want to seize the moment and make sure we coordinate.
I always wondered how this works in practice for "real time" use cases because we've seen with secure boot + tpm that we can attest that the boot was genuine at some point in the past, what about modifications that can happen after that?
As per the announcement, we’ll be building this over the next months and sharing more information as this rolls out. Much of the fundamentals can be extracted from Lennart’s posts and the talks from All Systems Go! over the last years.
One of the most grating pain points of the early versions of systemd was a general lack of humility, some would say rank arrogance, displayed by the project lead and his orbiters. Today systemd is in a state of "not great, not terrible" but it was (and in some circles still is) notorious for breaking peoples' linux installs, their workflows, and generally just causing a lot of headaches. The systemd project leads responded mostly with Apple-style "you're holding it wrong" sneers.
It's not immediately clear to me what exactly Amutable will be implementing, but it smells a lot like some sort of DRM, and my immediate reaction is that this is something that Big Tech wants but that users don't.
My question is this: Has Lennart's attitude changed, or can linux users expect more of the same paternalism as some new technology is pushed on us whether we like it or not?
You won't believe how many hours we have lost troubleshooting SysV init and Upstart issues. systemd is so much better in every way, reliable parallel init with dependencies, proper handling of double forking, much easier to secure services (systemd-analyze security), proper timer handling (yay, no more cron), proper temporary file/directory handling, centralized logs, etc.
It improves on about every level compared to what came before. And no, nothing is perfect and you sometimes have to troubleshoot it.
About ten years ago I took a three day cross-country Amtrak trip where I wanted to work on some data analysis that used mysql for its backend. It was a great venue for that sort of work because the lack of train-internet was wonderful to keep me focused. The data I was working with was about 20GB of parking ticket data. The data took a while to process over SQL which gave me the chance to check out the world unfolding outside of the train.
At some point, mysql (well, mariadb) got into a weird state after an unclean shutdown that put itself into recovery mode where upon startup it had to do some disk-intensive cleanup. Thing is -- systemd has a default setting (that's not readily documented, nor sufficiently described in its logs when the behavior happens) that halts the service startup after 30 seconds to try again. On loop.
My troubleshooting attempts were unsuccessful. And since I deleted the original csv files to save disk space, I wasn't able to even poke at the CSV files through python or whatnot.
So instead of doing the analysis I wanted to do on the train, I had to wait until I got to the end of the line to fix it. Sure enough, it was some default 30s timeout that's not explicitly mentioned nor commented out like many services do.
So, saying that things are "much better in every way" really falls on deaf ears and is reminiscent of the systemd devs' dismissive/arrogant behavior that many folk are frustrated about.
How can I cancel a systemd startup task that blocks the login prompt? / how is forcing me to wait for dhcp on a network interface that isn't even plugged in a better experience?
Your distribution has configured your GDM or Getty to have some dependency on something that ultimately waits on dhcpcd/network-online.target.
It’s not really the fault of systemd; it just enables new possibilities that were previously difficult/impossible and now the usage of said possibilities is surfacing problems.
It is the fault of systemd that there's no interactive control.
On other inits, I can hit ctrl-C to break out of a poorly configured setup. Yes, it's more difficult when there's potentially parallelism. But systemd is not uniformly better than everything else when it lacks interactivity.
And it might not be better than everything else if common distributions set it up wrong because it's difficult to set it up right. If we're willing to discount problems related to one init system because the distribution is holding it wrong, then why don't we blame problems with other init systems on distributions or applications, too? There's no need to restart crashing applications if applications don't crash, etc.
There’s a reason why Devuan (a non systemd Debian) exists. Don’t want to get into a massive argument, but there are legitimate reasons for some to go in a different direction.
After over a decade of Debian, when I upgraded my PC, I tried every big systemd-based distro, including opensuse, which I wholly loathed. I finally decided on Void and feel at home as I did 20+ years ago when I began.
There are serious problems with the systemd paradigm, most of which I couldn't argue for or against. But at least in Void, I can remove network-manger altogether, use cron as I always have, and generally remain free to do as I please until eventually every package there is has systemd dependencies which seems frightfully plausible at this pace.
Void is as good as I could have wanted. If that ever goes, I guess it's either BSD or a cave somewhere.
I'm glad to see the terse questions here. They're well warranted.
Here are a few examples of problems systemd has caused me:
System shutdown/reboot is now unreliable. Sometimes it will be just as quick as it was before systemd arrived, but other times, systemd will decide that something isn't to its liking, and block shutdown for somewhere between 30 seconds and 10 minutes, waiting for something that will never happen. The thing in question might be different from one session to the next, and from one systemd version to the next; I can spend hours or days tracking down the process/mount/service in question and finding a workaround, only to have systemd hang on something else the next day. It offers no manual skip option, so unless I happen to be working on a host with systemd's timeouts reconfigured to reduce this problem, I'm stuck with either forcing a power-off or having my time wasted.
Something about systemd's meddling with cgroups broke the lxc control commands a few years back. To work around the problem, I have to replace every such command I use with something like `systemd-run --quiet --user --scope --property=Delegate=yes <command>`. That's a PITA that I'm unlikely to ever remember (or want to type) so I effectively cannot manage containers interactively without helper scripts any more. It's also a new systemd dependency, so those helper scripts now also need checks for cgroup version and systemd presence, and a different code path depending on the result. Making matters worse, that systemd-run command occasionally fails even when I do everything "right". What was once simple and easy is now complex and unreliable.
At some point, Lennart unilaterally decided that all machines accessed over a network must have a domain name. Subsequently, every machine running a distro that had migrated to systemd-resolved was suddenly unable to resolve its hostname-only peers on the LAN, despite the DNS server handling them just fine. Finding the problem, figuring out the cause, and reconfiguring around it wasn't the end of the world, but it did waste more of my time. Repeating that experience once or twice more when systemd behavior changed again and again eventually drove me to a policy of ripping out systemd-resolved entirely on any new installation. (Which, of course, takes more time.) I think this behavior may have been rolled back by now, but sadly, I'll never get my time back.
There are more examples, but I'm tired of re-living them and don't really want to write a book. I hope these few are enough to convey my point:
Systemd has been a net negative in my experience. It has made my life markedly worse, without bringing anything I needed. Based on conversations, comments, and bug reports I've seen over the years, I get the impression that many others have had a similar experience, but don't bother speaking up about it any more, because they're tired of being dismissed, ignored, or shouted down, just as I am.
I would welcome a reliable, minimal, non-invasive, dependency-based init. Systemd is not it.
The problem is not systemd vs SysV et al, the problem is systemd spreading like a cancer throughout the entire operating system.
Also trying to use systemd with podman is frustrating as hell. You just cannot run a system service using podman as a non-root user and have it work correctly.
Quadlet actually solves this. It's the newer way to define containers for systemd and handles the rootless user case properly. I migrated my services to it recently and it's much more robust than the old generate scripts.
Quadlet are great but running podman via systemd as a non root user worked perfectly well before quadlets and I have no idea what your parent is talking about (I'm currently in the process of converting my home services from rootless podman over systemd to quadlet)
Why would you go to the effort to present faces on an "About Us" page when its only white men anyway ? Do you think Women and PoCs are incompatible with your vision of saving the Linux (not GNU, you don't mention GNU in your mission statement at all) World through germanic rigor ? Genuinely curious because you are based in Berlin where I have never been to a start-up office or techno club that wasn't at least as diverse as its SF or NYC equivalent. If you were based in Munich or Stuttgart this would be easier to pass off as a curious coincidence.
I'm sorry that you find pictures of prominent Linux kernel and core systemd maintainers unsatisfactory. What do you propose they do about their appearance?
You'd probably best ask a stylist about that. I was pointing at a different issue. Historically (at least in the last decade or so) it became common best practice in such governance focused projects pushing systemic changes in large and diverse projects to aim for a minimum of inclusivity at least in "community / developer outreach" type roles and not to seed an office culture with only members of a single ethnicity and gender.
Yea excuse my autism its making me slower to adapt to a changing reality where diversity and inclusion has suddenly become irrelevant even in projects focused on issues pertaining to Linux governance and accessibility. I'm still mentally stuck in the early 2020s when every FOSS Project had BLM banners on their site pronouncing their "deep dedication" to female and minority empowerment and I thought after the last election cycle we'd now be back at the "equal opportunities" stage and hadn't regressed to the "females have periods and blacks look funny and having them in the office kills the bro-vibes and means you can only do racist and sexist jokes at home" stage yet... Obviously I was wrong and the only thing that matters is whether and what they have raised and at what valuation. It's also great that they are not doing that soft AI stuff but extremely erect hard crypto(graphy not the other kind) which is, needless to say, only possible with the manliest of male alpha-turing Chads.
The immediate concern seeing this is will the maintainer of systemd use their position to push this on everyone through it like every other extended feature of systemd?
Whatever it is, I hope it doesn't go the usual path of a minimal support, optional support and then being virtually mandatory by means of tight coupling with other subsystems.
Daan here, founding engineer and systemd maintainer.
So we try to make every new feature that might be disruptive optional in systemd and opt-in. Of course we don't always succeed and there will always be differences in opinion.
Also, we're a team of people that started in open source and have done open source for most of our careers. We definitely don't intend to change that at all. Keeping systemd a healthy project will certainly always stay important for me.
Thanks for the answer. Let me ask you something close with a more blunt angle:
Considering most of the tech is already present and shipping in the current systemd, what prevents our systems to become a immutable monolith like macOS or current Android with the flick of a switch?
Or a more grave scenario: What prevents Microsoft from mandating removal of enrollment permissions for user keychains and Secure Boot toggle, hence every Linux distribution has to go through Microsoft's blessing to be bootable?
So adding all of this technology will certainly make it more easy to be used for either good or bad. And it will certainly become possible to build an OS that will be less hackable than your run of the mill Linux distro.
But we will never enforce using any of these features in systemd itself. It will always be up to the distro to enable and configure the system to become an immutable monolith. And I certainly don't think distributions like Fedora or Debian will ever go in that direction.
We don't really have any control over what Microsoft decides to do with Secure Boot. If they decide at one point to make Secure Boot reject any Linux distribution and hardware vendors prevent enrolling user owned keys, we're in just as much trouble as everyone else running Linux will be.
I doubt that will actually happen in practice though.
I would be _shocked_ if, conditional on your project being successful, this _wasn't_ commonly used to lock down computing abilities commonly taken for granted today. And I think you know this.
Also, I know. A few of my colleagues run {open, free, dragonfly}BSD as their daily drivers for more than two decades. Also, we have BSD based systems at a couple of places.
However, as a user of almost all mainstream OSes (at the same time, for different reasons), and planning to include OpenBSD to that roster (taking care of a fleet takes time), I'd love to everyone select the correct tool for their applications and don't throw stones at people who doesn't act like them.
Please remember that we all sit in houses made of glass before throwing things to others.
Oh, also please don't make assumptions about people you don't know.
You could describe Richard Stallman as someone who refuses to use proprietary software because he sees using it as becoming complicit--however indirectly--in a technology ecosystem that violates the values he’s committed to.
"Just don't use X" is in fact a very engaged and principled response. Try again.
If you were not a systemd maintainer and have started this project/company independently targeting systemd, you would have to go through the same process as everyone and I would have expected the systemd maintainers to, look at it objectively and review with healthy skepticism before accepting it. But we cannot rely on that basic checks and balances anymore and that's the most worrying part.
> that might be disruptive optional in systemd
> we don't always succeed and there will always be differences in opinion.
You (including other maintainers) are still the final arbitrator of what's disruptive. The differences of opinion in the past have mostly been settled as "deal with it" and that's the basis of current skepticism.
Systemd upstream has reviewers and maintainers from a bunch of different companies, and some independent: Red Hat, Meta, Microsoft, even some independent developers, etc. This isn't changing, we'll continue to work through consensus of maintainers regardless of which company we work at.
Drive encryption is only really securing your data at rest, not while the system is running. Ideally image based systems also use the kernels runtime integrity checking (e.g. dm-verity) to ensure that things are as they are expected to be.
“ensure that things are as they are expected to be” according to who, and for who's benefit? Certainly not the person sitting in front of the computer.
The system owner. Usually that is the same entity that owns the secure boot keys, which can be the person that bought a device or another person if the buyer decides to delegate that responsibility (whether knowingly or unknowingly).
In my case I am talking about myself. I prefer to actually know what is running on my systems and ensure that they are as I expect them to be and not that they may have been modified unbeknownst to me.
Remote attestation is another technology that is not inherently restrictive of software freedom. But here are some examples of technologies that have already restricted freedom due to oligopoly combined with network effects:
* smartphone device integrity checks (SafetyNet / Play Integrity / Apple DeviceCheck)
It very clearly is restrictive of software freedom. I've never suffered from an evil maid breaking into my house to access my computer, but I've _very_ frequently suffered from corporations trying to prevent me from doing what I wish with my own things. We need to push back on this notion that this sort of thing was _ever_ for the end-user's benefit, because it's not.
I am quite conflicted here. On one hand I understand the need for it (offsite colo servers is the best example). Basic level of evil maid resistance is also a nice to have on personal machines. On the other hand we have all the things you listed.
I personally don't think this product matters all that much for now. These types of tech is not oppressive by itself, only when it is being demanded by an adversary. The ability of the adversary to demand it is a function of how widespread the capability is, and there aren't going to be enough Linux clients for this to start infringing on the rights of the general public just yet.
A bigger concern is all the efforts aimed at imposing integrity checks on platforms like the Web. That will eventually force users to make a choice between being denied essential services and accepting these demands.
I also think AI would substantially curtail the effect of many of these anti-user efforts. For example a bot can be programmed to automate using a secure phone and controlled from a user-controlled device, cheat in games, etc.
Secure Boot allows you to enroll your own keys. This is part of the spec, and there are no shipped firmwares that prevents you from going through this process.
The authors clearly don’t intend this to happen but that doesn’t matter. Someone else will do it. Maybe this can be stopped with licensing as we tried to stop the SaaS loophole with GPLv3?
My only experience with Linux secure boot so far.... I wasn't even aware that it was secure booted. And I needed to run something (I think it was the Displaylink driver) that needs to jam itself into the kernel. And the convoluted process to do it failed (it's packaged for Ubuntu but I was installing it on a slightly outdated Fedora system).
What, this part is only needed for secure boot? I'm not sec... oh. So go back to the UEFI settings, turn secure boot off, problem solved. I usually also turn off SELinux right after install.
So I'm an old greybeard who likes to have full control. Less secure. But at least I get the choice. Hopefully I continue to do so. The notion of not being able to access online banking services or other things that require account login, without running on a "fully attested" system does worry me.
Secure Boot only extends the chain of trust from your firmware down the first UEFI binary it loads.
Currently SB is effectively useless because it will at best authenticate your kernel but the initrd and subsequent userspace (including programs that run as root) are unverified and can be replaced by malicious alternatives.
Secure Boot as it stands right now in the Linux world is effectively an annoyance that’s only there as a shortcut to get distros to boot on systems that trust Microsoft’s keys but otherwise offer no actual security.
It however doesn’t have to be this way, and I welcome efforts to make Linux just as secure as proprietary OSes who actually have full code signature verification all the way down to userspace.
It sounds like you want to achieve system transparency, but I don't see any clear mention of reproducible builds or transparency logs anywhere.
I have followed systemd's efforts into Secure Boot and TPM use with great interest. It has become increasingly clear that you are heading in a very similar direction to these projects:
- Hal Finney's transparent server
- Keylime
- System Transparency
- Project Oak
- Apple Private Cloud Compute
- Moxie's Confer.to
I still remember Jason introducing me to Lennart at FOSDEM in 2020, and we had a short conversation about System Transparency.
I'd love to meet up at FOSDEM. Email me at fredrik@mullvad.net.
Edit: Here we are six years later, and I'm pretty sure we'll eventually replace a lot of things we built with things that the systemd community has now built. On a related note, I think you should consider using Sigsum as your transparency log. :)
Edit2: For anyone interested, here's a recent lightning talk I did that explains the concept that all project above are striving towards, and likely Amutable as well: https://www.youtube.com/watch?v=Lo0gxBWwwQE
Our entire team will be at FOSDEM, and we'd be thrilled to meet more of the Mullvad team. Protecting systems like yours is core to us. We want to understand how we put the right roots of trust and observability into your hands.
Edit: I've reached out privately by email for next steps, as you requested.
Hi David. Great! I actually wasn't planning on going due to other things, but this is worth re-arranging my schedule a bit. See you later this week. Please email me your contact details.
As I mentioned above, we've followed systemd's development in recent years with great interest, as well as that of some other projects. When I started(*) the System Transparency project it was very much a research project.
Today, almost seven years later, I think there's a great opportunity for us to reduce our maintenance burden by re-architecting on top of systemd, and some other things. That way we can focus on other things. There's still a lot of work to do on standardizing transparency building blocks, the witness ecosystem(**), and building an authentication mechanism for system transparency that weaves it all together.
I'm more than happy to share my notes with you. Best case you build exactly what we want. Then we don't have to do it. :)
I'm super far from an expert on this, but it NEEDS reproducible builds, right? You need to start from a known good, trusted state - otherwise you cannot trust any new system states. You also need it for updates.
Well, it comes down to what trust assumptions you're OK with. Reproducible reduces trust in the build environment, but you still need to ensure authenticity of the source somehow. Verified boot, measured boot, repro builds, local/remote attestation, and transparency logging provide different things. Combined they form the possibility of a sort of authentication mechanism between a server and client. However, all of the concepts are useful by themselves.
Probably obvious from the surnames but this is the first time I've seen a EU company pop up on Hacker News that could be mistaken for a Californian company. Nice to see that ambition.
I understand systemd is controversial, that can be debated endlessly but the executive team and engineering team look very competitive. Will be interesting to see where this goes.
Lennart will be involved with at least three events at FOSDEM on the coming weekend. The talks seem unrelated at first glance but maybe there will be an opportunity to learn more about his new endeavor.
systemd solved/improved a bunch of things for linux, but now the plan seems to be to replace package management with image based whole dist a/b swaps. and to have signed unified kernel images.
this basically will remove or significantly encumber user control over their system, such that any modification will make you loose your "signed" status and ... boom! goodbye accessing the internet without an id
pottering recently works for Microsoft, they want to turn linux into an appliance just like windows, no longer a general purpose os. the transition is still far from over on windows, but look at android and how the google play services dependency/choke-hold is
im sure ill get many down votes, but despite some hyperbole this is the trajectory
Immutability means you can't touch or change some parts of the system without great effort (e.g. macOS SIP).
Atomicity means you can track every change, and every change is so small that it affects only one thing and can be traced, replayed or rolled back. Like it's going from A to B and being able to return back to A (or going to B again) in a determinate manner.
Just an assumption here, but the project appears to be about the methodology to verify the install. Who holds the keys is an entirely different matter.
I'm sure this company is more focused on the enterprise angle, but I wonder if the buildout of support for remote attestation could eventually resolve the Linux gaming vs. anti-cheat stalemate. At least for those willing to use a "blessed" distro.
I might be behind on the latest counter-counter-counter-measures, but I know some of the leading AC solutions are already using IOMMU to wedge a firewall between passive DMA sniffers and the game processes memory.
rust-vmm-based environment that verifies/authenticates an image before running ? Immutable VM (no FS, root dropper after setting up network, no or curated device), 'micro'-vm based on systemd ? vmm captures running kernel code/memory mapping before handing off to userland, checks periodically it hasn't changed ? Anything else on the state of the art of immutable/integrity-checking of VMs?
There are genuine positive applications for remote attestation. E.g., if you maintain a set of servers, you can verify that it runs the software it should be running (the software is not compromised). Or if you are running something similar to Apple's Private Compute Cloud to run models, users can verify that it is running the privacy-preserving image that it is claiming to be running.
There are also bad forms of remote attestation (like Google's variant that helps them let banks block you if you are running an alt-os). Those suck and should be rejected.
Hacker News has recently been dominated by conspiracy theorists who believe that all applications of cryptography are evil attempts by shadowy corporate overlords to dominate their use of computing.
Buddy, if I want encryption of my own I've got secure boot, LUKS, GPG, etc. With all of those, why would I need or even want remote attestation? The purpose of that is to assure corporations that their code is running on my computer without me being able to modify it. It's for DRM.
I am fairly confident that this company is going to assure corporations that their own code is running on their own computers (ie - to secure datacenter workloads), to allow _you_ (or auditors) to assure that only _your_ asserted code is also running on their rented computers (to secure cloud workloads), or to assure that the code running on _their_ computers is what they say it is, which is actually pretty cool since it lets you use Somebody Else's Computer with some assurance that they aren't spying on you (see: Apple Private Cloud Compute). Maybe they will also try to use this to assert "deep" embedded devices which already lock the user out, although even this seems less likely given that these devices frequently already have such systems in place.
IMO it's pretty clear that this is a server play because the only place where Linux has enough of a foothold to make client / end-user attestation financially interesting is Android, where it already exists. And to me the server play actually gives me more capabilities than I had: it lets me run my code on cloud provided machines and/or use cloud services with some level of assurance that the provider hasn't backdoored me and my systems haven't been compromised.
How can you be "pretty sure" they're going to develop precisely the technology needed to implement DRM but also will never use or allow it to be used by anybody but the lawful owners of the hardware? You can't.
The typical HN rage-posting about DRM aside, there's no reason that remote attestation can't be used in the opposite direction: to assert that a server is running only the exact code stack it claims to be, avoiding backdoors. This can even be used with fully open-source software, creating an opportunity for OSS cloud-hosted services which can guarantee that the OSS and the build running on the server match. This is a really cool opportunity for privacy advocates if leveraged correctly - the idea could be used to build something like Apple's Private Cloud Compute but even more open.
Like evil maid attacks, this is a vanishingly rare scenario brought out to try to justify technology that will overwhelmingly be used to restrict computing freedom.
> but considering Windows requirements drive the PC spec, this capability can be used to force Linux distributions in bad ways
What do you mean by this?
Is the concern that systemd is suddenly going to require that users enable some kind of attestation functionality? That making attestation possible or easier is going to cause third parties to start requiring it for client machines running Linux? This doesn't even really seem to be a goal; there's not really money to be made there.
As far as I can tell the sales pitch here is literally "we make it so you can assure the machines running in your datacenter are doing what they say they are," which seems pretty nice to me, and the perversions of this to erode user rights are either just as likely as they ever were or incredibly strange edge cases.
Microsoft has a "minimum set of requirements" document about "Designed for Windows" PCs. You can't sell a machine with Windows or tell it's Windows compatible without complying with that checklist.
So, every PC sold to consumers is sanctioned by Microsoft. This list contains Secure Boot and TPM based requirements, too.
If Microsoft decides to eliminate enrollment of user keys and Secure Boot toggle, they can revoke current signing keys for "shims" and force Linux distributions to go full immutable to "sign" their bootloaders so they can boot.
As said above, it's not something Amutable can control, but enable by proxy and by accident.
Look, I work in a datacenter, with a sizeable fleet. Being able to verify that fleet is desirable for some kinds of operations, I understand that. On the other hand, like every double edged sword, this can cut in both ways.
I don't see how this relates in any way to Amutable and it has been a "concern" for 20+ years (which has never come to pass). How do you think this relates at all?
Before this point in time, Linux never supported being an immutable image. Neither filesystems, nor the mechanism to lock it down was there. The best you could do was, TiVoization, but that would be too obvious and won't fly.
Now we have immutable distributions (SuSE, Fedora, NixOS). We have the infrastructure for attestation (systemd's UKI, image based boot, and other immutability features), TPMs and controversially uutils (Which is MIT licensed and has the stated goal to replace all GNU userspace).
You can build an immutable and adversarial userspace where you don't have to share the source, and require every boot and application call to attest. The theoretical thickness of the wall is both much greater and this theoretical state is much easier to achieve.
20 years ago the only barrier was booting. After that everything was free. Now it's possible to boot into a prison where your every ls and cd command can be attested.
> Before this point in time, Linux never supported being an immutable image.
What? As just one example, dm-verity was merged into the mainline kernel 13 years ago. I built immutable, verified Linux systems at least ten years ago, and it was considered old hat by the time I got there.
> The best you could do was, TiVoization, but that would be too obvious and won't fly.
What does this even mean? "TiVoization" is the slang for "you get a device that runs Linux, you get the GPL sources, but you can't flash your own image on the device because you don't own the keys." This is the exact same problem then as it was now and just as "obvious?"
I understand the fears that come from client attestation (certainly, the way it has been used on Android has been majorly detrimental to non-Google ROMs), but, to the Android point, the groundwork has always been there.
I'd be very annoyed if someone showed up and said "we're making a Linux-based browser attestation system that your bank is going to partner on," but nobody has even gone this direction on Windows yet.
> Oh, Rust is memory safe. Good luck finding holes.
I break secure boot systems for a living and I'd say _maybe_ half of the bugs I find relate to memory safety in a way Rust would fix. A lot of systems already use tools which provide very similar safety guarantees to Rust for single threaded code. Systems are definitely getting more secure and I do worry about impenetrable fortresses appearing in the near future, but making this argument kind of undermines credibility in this space IMO.
Hi, Chris here, CEO @ Amutable. We are very excited about this. Happy to answer questions.
This seems like the kind of technology that could make the problem described in https://www.gnu.org/philosophy/can-you-trust.en.html a lot worse. Do you have any plans for making sure it doesn't get used for that?
I'm Aleksa, one of the founding engineers. We will share more about this in the coming months but this is not the direction nor intention of what we are working on. The models we have in mind for attestation are very much based on users having full control of their keys. This is not just a matter of user freedom, in practice being able to do this is far more preferable for enterprises with strict security controls.
I've been a FOSS guy my entire adult life, I wouldn't put my name to something that would enable the kinds of issues you describe.
Thanks, this would be helpful. I will follow on by recommending that you always make it a point to note how user freedom will be preserved, without using obfuscating corpo-speak or assuming that users don’t know what they want, when planning or releasing products. If you can maintain this approach then you should be able to maintain a good working relationship with the community. If you fight the community you will burn a lot of goodwill and will have to spend resources on PR. And there is only so much that PR can do!
Better security is good in theory, as long as the user maintains control and the security is on the user end. The last thing we need is required ID linked attestation for accessing websites or something similar.
Thanks for the reassurance, the first ray of sunshine in this otherwise rather alarming thread. Your words ring true.
It would be a lot more reassuring if we knew what the business model actually was, or indeed anything else at all about this. I remain somewhat confused as to the purpose of this announcement when no actual information seems to be forthcoming. The negative reactions seen here were quite predictable, given the sensitive topic and the little information we do have.
half of the founders of this thing come from Microsoft. I suppose this makes the answer to your question obvious.
that's a silver lining
the anti-user attestation will at least be full of security holes, and likely won't work at all
My thoughts exactly. We're probably witnessing the beginning of the end of linux users being able to run their own kernels. Soon:
- your bank won't let you log in from an "insecure" device.
- you won't be able to play videos on an "insecure" device.
- you won't be able to play video games on an "insecure" device.
And so on, and so forth.
This is already the world we live in when it comes to the most popular personal computing devices running Linux out there.
Really excited to a company investing into immutable and cryptographically verifiable systems. Two questions really:
1. How will the company make money? (You have probably been asked that a million times :).)
2. Similar to the sibling: what are the first bits that you are going to work on.
At any rate, super cool and very nice that you are based in EU/Germany/Berlin!
1. We are confident we have a very robust path to revenue.
2. Given the team, it should be quite obvious there will be a Linux-based OS involved.
Our aims are global but we certainly look forward to playing an important role in the European tech landscape.
"We are confident we have a very robust path to revenue."
I take it that you are not at this stage able to provide details of the nature of the path to revenue. On what kind of timescale do you envisage being able to disclose your revenue stream/subscribers/investors?
"Ubuntu Core" is a similar product [1]
As I understand it, the main customers for this sort of thing are companies making Tivo-style products - where they want to use Linux in their product, but they want to lock it down so it can't be modified by the device owner.
This can be pretty profitable; once your customers have rolled out a fleet of hardware locked down to only run kernels you've signed.
[1] https://ubuntu.com/core
This sounds like a net negative for the end user
How do you plan handle the confused deputy problem?[1]
[1] https://en.wikipedia.org/wiki/Confused_deputy_problem
"We are building cryptographically verifiable integrity into Linux systems. Every system starts in a verified state and stays trusted over time."
What does this mean? Why would anyone want this? Can you explain this to me like I'm five years old?
https://youtu.be/EzSkU3Oecuw?si=1fNV6XkyTv7SfpJs
Do you plan to sell this technology to laptop makers so their laptops will only run the OS they came with?
Or, worse, run any unsupported linux as long as it contains systemd, so no *bsd, etc, and also no manufacturer support?
Will this do remote attestation ? What hardware platforms will it support? (Intel sgx, AMD sev, AWS nitro?)
1. Are reproducible builds and transparency logging part of your concept?
2. Are you looking for pilot customers?
Damn, you are thirsty!
Are these some problems you've personally been dealing with?
I just want more trustworthy systems. This particular concept of combining reproducible builds, remote attestation and transparency logs is something I came up with in 2018. My colleagues and I started working on it, took a detour into hardware (tillitis.se) and kind of got stuck on the transparency part (sigsum.org, transparency.dev, witness-network.org).
Then we discovered snapshot.debian.org wasn't feeling well, so that was another (important) detour.
Part of me wish we had focused more on getting System Transparency in its entirety in production at Mullvad. On the other hand I certainly don't regret us creating Tillitis TKey, Sigsum, taking care of Debian Snapshot service, and several other things.
Now, six years later, systemd and other projects have gotten a long way to building several of the things we need for ST. It doesn't make sense to do double work, so I want to seize the moment and make sure we coordinate.
I always wondered how this works in practice for "real time" use cases because we've seen with secure boot + tpm that we can attest that the boot was genuine at some point in the past, what about modifications that can happen after that?
Can you share more details at this point about what you are trying to tackle as a first step?
As per the announcement, we’ll be building this over the next months and sharing more information as this rolls out. Much of the fundamentals can be extracted from Lennart’s posts and the talks from All Systems Go! over the last years.
I'm sorry, you're "happy to answer questions" and this is your reply to such a softball? What kind of questions will you answer? Favorite color?
fantastic news, congrats on launching! it's a great mission statement a fanstastic ensemble for the job
Hi Chris,
One of the most grating pain points of the early versions of systemd was a general lack of humility, some would say rank arrogance, displayed by the project lead and his orbiters. Today systemd is in a state of "not great, not terrible" but it was (and in some circles still is) notorious for breaking peoples' linux installs, their workflows, and generally just causing a lot of headaches. The systemd project leads responded mostly with Apple-style "you're holding it wrong" sneers.
It's not immediately clear to me what exactly Amutable will be implementing, but it smells a lot like some sort of DRM, and my immediate reaction is that this is something that Big Tech wants but that users don't.
My question is this: Has Lennart's attitude changed, or can linux users expect more of the same paternalism as some new technology is pushed on us whether we like it or not?
Thank you for this question, it perfectly captures something that I believe many would like answered.
As someone who's lost many hours troubleshooting systemd failures, I would like an answer to this question, too.
You won't believe how many hours we have lost troubleshooting SysV init and Upstart issues. systemd is so much better in every way, reliable parallel init with dependencies, proper handling of double forking, much easier to secure services (systemd-analyze security), proper timer handling (yay, no more cron), proper temporary file/directory handling, centralized logs, etc.
It improves on about every level compared to what came before. And no, nothing is perfect and you sometimes have to troubleshoot it.
"In every way"
About ten years ago I took a three day cross-country Amtrak trip where I wanted to work on some data analysis that used mysql for its backend. It was a great venue for that sort of work because the lack of train-internet was wonderful to keep me focused. The data I was working with was about 20GB of parking ticket data. The data took a while to process over SQL which gave me the chance to check out the world unfolding outside of the train.
At some point, mysql (well, mariadb) got into a weird state after an unclean shutdown that put itself into recovery mode where upon startup it had to do some disk-intensive cleanup. Thing is -- systemd has a default setting (that's not readily documented, nor sufficiently described in its logs when the behavior happens) that halts the service startup after 30 seconds to try again. On loop.
My troubleshooting attempts were unsuccessful. And since I deleted the original csv files to save disk space, I wasn't able to even poke at the CSV files through python or whatnot.
So instead of doing the analysis I wanted to do on the train, I had to wait until I got to the end of the line to fix it. Sure enough, it was some default 30s timeout that's not explicitly mentioned nor commented out like many services do.
So, saying that things are "much better in every way" really falls on deaf ears and is reminiscent of the systemd devs' dismissive/arrogant behavior that many folk are frustrated about.
> systemd is so much better in every way,
How can I cancel a systemd startup task that blocks the login prompt? / how is forcing me to wait for dhcp on a network interface that isn't even plugged in a better experience?
Your distribution has configured your GDM or Getty to have some dependency on something that ultimately waits on dhcpcd/network-online.target.
It’s not really the fault of systemd; it just enables new possibilities that were previously difficult/impossible and now the usage of said possibilities is surfacing problems.
It is the fault of systemd that there's no interactive control.
On other inits, I can hit ctrl-C to break out of a poorly configured setup. Yes, it's more difficult when there's potentially parallelism. But systemd is not uniformly better than everything else when it lacks interactivity.
And it might not be better than everything else if common distributions set it up wrong because it's difficult to set it up right. If we're willing to discount problems related to one init system because the distribution is holding it wrong, then why don't we blame problems with other init systems on distributions or applications, too? There's no need to restart crashing applications if applications don't crash, etc.
There’s a reason why Devuan (a non systemd Debian) exists. Don’t want to get into a massive argument, but there are legitimate reasons for some to go in a different direction.
And Void Linux. And Gentoo. And Alpine Linux. And Slackware. And others.
Systemd has recently added experimental support for musl libc, which should eventually allow Alpine to upgrade though
After over a decade of Debian, when I upgraded my PC, I tried every big systemd-based distro, including opensuse, which I wholly loathed. I finally decided on Void and feel at home as I did 20+ years ago when I began.
There are serious problems with the systemd paradigm, most of which I couldn't argue for or against. But at least in Void, I can remove network-manger altogether, use cron as I always have, and generally remain free to do as I please until eventually every package there is has systemd dependencies which seems frightfully plausible at this pace.
Void is as good as I could have wanted. If that ever goes, I guess it's either BSD or a cave somewhere.
I'm glad to see the terse questions here. They're well warranted.
How is systemd stopping you use cron?
Here are a few examples of problems systemd has caused me:
System shutdown/reboot is now unreliable. Sometimes it will be just as quick as it was before systemd arrived, but other times, systemd will decide that something isn't to its liking, and block shutdown for somewhere between 30 seconds and 10 minutes, waiting for something that will never happen. The thing in question might be different from one session to the next, and from one systemd version to the next; I can spend hours or days tracking down the process/mount/service in question and finding a workaround, only to have systemd hang on something else the next day. It offers no manual skip option, so unless I happen to be working on a host with systemd's timeouts reconfigured to reduce this problem, I'm stuck with either forcing a power-off or having my time wasted.
Something about systemd's meddling with cgroups broke the lxc control commands a few years back. To work around the problem, I have to replace every such command I use with something like `systemd-run --quiet --user --scope --property=Delegate=yes <command>`. That's a PITA that I'm unlikely to ever remember (or want to type) so I effectively cannot manage containers interactively without helper scripts any more. It's also a new systemd dependency, so those helper scripts now also need checks for cgroup version and systemd presence, and a different code path depending on the result. Making matters worse, that systemd-run command occasionally fails even when I do everything "right". What was once simple and easy is now complex and unreliable.
At some point, Lennart unilaterally decided that all machines accessed over a network must have a domain name. Subsequently, every machine running a distro that had migrated to systemd-resolved was suddenly unable to resolve its hostname-only peers on the LAN, despite the DNS server handling them just fine. Finding the problem, figuring out the cause, and reconfiguring around it wasn't the end of the world, but it did waste more of my time. Repeating that experience once or twice more when systemd behavior changed again and again eventually drove me to a policy of ripping out systemd-resolved entirely on any new installation. (Which, of course, takes more time.) I think this behavior may have been rolled back by now, but sadly, I'll never get my time back.
There are more examples, but I'm tired of re-living them and don't really want to write a book. I hope these few are enough to convey my point:
Systemd has been a net negative in my experience. It has made my life markedly worse, without bringing anything I needed. Based on conversations, comments, and bug reports I've seen over the years, I get the impression that many others have had a similar experience, but don't bother speaking up about it any more, because they're tired of being dismissed, ignored, or shouted down, just as I am.
I would welcome a reliable, minimal, non-invasive, dependency-based init. Systemd is not it.
The problem is not systemd vs SysV et al, the problem is systemd spreading like a cancer throughout the entire operating system.
Also trying to use systemd with podman is frustrating as hell. You just cannot run a system service using podman as a non-root user and have it work correctly.
> You just cannot run a system service using podman as a non-root user and have it work correctly.
Err... You just need to run `podman-compose systemd`?
I have my entire self-hosted stack running with systemd-controlled Podman, in regular user accounts.
Quadlet actually solves this. It's the newer way to define containers for systemd and handles the rootless user case properly. I migrated my services to it recently and it's much more robust than the old generate scripts.
Quadlet are great but running podman via systemd as a non root user worked perfectly well before quadlets and I have no idea what your parent is talking about (I'm currently in the process of converting my home services from rootless podman over systemd to quadlet)
I'll ask the dumb question sorry!
Who is this for / what problem does it solve?
I guess security? Or maybe reproducability?
Why would you go to the effort to present faces on an "About Us" page when its only white men anyway ? Do you think Women and PoCs are incompatible with your vision of saving the Linux (not GNU, you don't mention GNU in your mission statement at all) World through germanic rigor ? Genuinely curious because you are based in Berlin where I have never been to a start-up office or techno club that wasn't at least as diverse as its SF or NYC equivalent. If you were based in Munich or Stuttgart this would be easier to pass off as a curious coincidence.
I'm sorry that you find pictures of prominent Linux kernel and core systemd maintainers unsatisfactory. What do you propose they do about their appearance?
You'd probably best ask a stylist about that. I was pointing at a different issue. Historically (at least in the last decade or so) it became common best practice in such governance focused projects pushing systemic changes in large and diverse projects to aim for a minimum of inclusivity at least in "community / developer outreach" type roles and not to seed an office culture with only members of a single ethnicity and gender.
Who cares. That is all irrelevant.
I want to know if they raised VC money or not.
Either way at least it isn't anything about AI and has something to do with hard cryptography.
Yea excuse my autism its making me slower to adapt to a changing reality where diversity and inclusion has suddenly become irrelevant even in projects focused on issues pertaining to Linux governance and accessibility. I'm still mentally stuck in the early 2020s when every FOSS Project had BLM banners on their site pronouncing their "deep dedication" to female and minority empowerment and I thought after the last election cycle we'd now be back at the "equal opportunities" stage and hadn't regressed to the "females have periods and blacks look funny and having them in the office kills the bro-vibes and means you can only do racist and sexist jokes at home" stage yet... Obviously I was wrong and the only thing that matters is whether and what they have raised and at what valuation. It's also great that they are not doing that soft AI stuff but extremely erect hard crypto(graphy not the other kind) which is, needless to say, only possible with the manliest of male alpha-turing Chads.
The immediate concern seeing this is will the maintainer of systemd use their position to push this on everyone through it like every other extended feature of systemd?
Whatever it is, I hope it doesn't go the usual path of a minimal support, optional support and then being virtually mandatory by means of tight coupling with other subsystems.
Daan here, founding engineer and systemd maintainer.
So we try to make every new feature that might be disruptive optional in systemd and opt-in. Of course we don't always succeed and there will always be differences in opinion.
Also, we're a team of people that started in open source and have done open source for most of our careers. We definitely don't intend to change that at all. Keeping systemd a healthy project will certainly always stay important for me.
Hi Daan,
Thanks for the answer. Let me ask you something close with a more blunt angle:
Considering most of the tech is already present and shipping in the current systemd, what prevents our systems to become a immutable monolith like macOS or current Android with the flick of a switch?
Or a more grave scenario: What prevents Microsoft from mandating removal of enrollment permissions for user keychains and Secure Boot toggle, hence every Linux distribution has to go through Microsoft's blessing to be bootable?
So adding all of this technology will certainly make it more easy to be used for either good or bad. And it will certainly become possible to build an OS that will be less hackable than your run of the mill Linux distro.
But we will never enforce using any of these features in systemd itself. It will always be up to the distro to enable and configure the system to become an immutable monolith. And I certainly don't think distributions like Fedora or Debian will ever go in that direction.
We don't really have any control over what Microsoft decides to do with Secure Boot. If they decide at one point to make Secure Boot reject any Linux distribution and hardware vendors prevent enrolling user owned keys, we're in just as much trouble as everyone else running Linux will be.
I doubt that will actually happen in practice though.
I would be _shocked_ if, conditional on your project being successful, this _wasn't_ commonly used to lock down computing abilities commonly taken for granted today. And I think you know this.
Nothing, but openbsd is amazing and just works. Anyone still using Linux on the desktop in 2026 should switch.
"Just don't use X" doesn't solve any problems in any space, unfortunately.
Plus, it's an avoidant and reductionist take.
Note: I have nothing against BSDs, but again, this is not the answer.
It works for me and for millions of others.
Stop trying to make everyone act like you act.
I'm not trying to make everyone act like I act.
Also, I know. A few of my colleagues run {open, free, dragonfly}BSD as their daily drivers for more than two decades. Also, we have BSD based systems at a couple of places.
However, as a user of almost all mainstream OSes (at the same time, for different reasons), and planning to include OpenBSD to that roster (taking care of a fleet takes time), I'd love to everyone select the correct tool for their applications and don't throw stones at people who doesn't act like them.
Please remember that we all sit in houses made of glass before throwing things to others.
Oh, also please don't make assumptions about people you don't know.
> Stop trying to make everyone act like you act.
Yeah! Telling people what to do is rude!
> Anyone still using Linux on the desktop in 2026 should switch
Oh.
You could describe Richard Stallman as someone who refuses to use proprietary software because he sees using it as becoming complicit--however indirectly--in a technology ecosystem that violates the values he’s committed to.
"Just don't use X" is in fact a very engaged and principled response. Try again.
Thanks Daan for your contributions to systemd.
If you were not a systemd maintainer and have started this project/company independently targeting systemd, you would have to go through the same process as everyone and I would have expected the systemd maintainers to, look at it objectively and review with healthy skepticism before accepting it. But we cannot rely on that basic checks and balances anymore and that's the most worrying part.
> that might be disruptive optional in systemd
> we don't always succeed and there will always be differences in opinion.
You (including other maintainers) are still the final arbitrator of what's disruptive. The differences of opinion in the past have mostly been settled as "deal with it" and that's the basis of current skepticism.
Systemd upstream has reviewers and maintainers from a bunch of different companies, and some independent: Red Hat, Meta, Microsoft, even some independent developers, etc. This isn't changing, we'll continue to work through consensus of maintainers regardless of which company we work at.
>We are building cryptographically verifiable integrity into Linux systems. Every system starts in a verified state and stays trusted over time.
What problem does this solve for Linux or people who use Linux? Why is this different from me simply enabling encryption on the drive?
Drive encryption is only really securing your data at rest, not while the system is running. Ideally image based systems also use the kernels runtime integrity checking (e.g. dm-verity) to ensure that things are as they are expected to be.
“ensure that things are as they are expected to be” according to who, and for who's benefit? Certainly not the person sitting in front of the computer.
The system owner. Usually that is the same entity that owns the secure boot keys, which can be the person that bought a device or another person if the buyer decides to delegate that responsibility (whether knowingly or unknowingly).
In my case I am talking about myself. I prefer to actually know what is running on my systems and ensure that they are as I expect them to be and not that they may have been modified unbeknownst to me.
This is only the case if the person sitting in front of it does not own the keys.
It prevents malware that obtained root access once from forever replacing your kernel/initrd and achieving persistence that way.
systemd is the most well supported init systemd there.
Remote attestation is another technology that is not inherently restrictive of software freedom. But here are some examples of technologies that have already restricted freedom due to oligopoly combined with network effects:
* smartphone device integrity checks (SafetyNet / Play Integrity / Apple DeviceCheck)
* HDMI/HDCP
* streaming DRM (Widevine / FairPlay)
* Secure Boot (vendor-keyed deployments)
* printers w/ signed/chipped cartridges (consumables auth)
* proprietary file formats + network effects (office docs, messaging)
It very clearly is restrictive of software freedom. I've never suffered from an evil maid breaking into my house to access my computer, but I've _very_ frequently suffered from corporations trying to prevent me from doing what I wish with my own things. We need to push back on this notion that this sort of thing was _ever_ for the end-user's benefit, because it's not.
I am quite conflicted here. On one hand I understand the need for it (offsite colo servers is the best example). Basic level of evil maid resistance is also a nice to have on personal machines. On the other hand we have all the things you listed.
I personally don't think this product matters all that much for now. These types of tech is not oppressive by itself, only when it is being demanded by an adversary. The ability of the adversary to demand it is a function of how widespread the capability is, and there aren't going to be enough Linux clients for this to start infringing on the rights of the general public just yet.
A bigger concern is all the efforts aimed at imposing integrity checks on platforms like the Web. That will eventually force users to make a choice between being denied essential services and accepting these demands.
I also think AI would substantially curtail the effect of many of these anti-user efforts. For example a bot can be programmed to automate using a secure phone and controlled from a user-controlled device, cheat in games, etc.
> * Secure Boot (vendor-keyed deployments)
I wish this myth would die at this point.
Secure Boot allows you to enroll your own keys. This is part of the spec, and there are no shipped firmwares that prevents you from going through this process.
> Secure Boot allows you to enroll your own keys
UEFI secure boot on PCs, yes for the most part. A lot of mobile platforms just never supported this. It's not a myth.
Phones don't implement UEFI.
The authors clearly don’t intend this to happen but that doesn’t matter. Someone else will do it. Maybe this can be stopped with licensing as we tried to stop the SaaS loophole with GPLv3?
My only experience with Linux secure boot so far.... I wasn't even aware that it was secure booted. And I needed to run something (I think it was the Displaylink driver) that needs to jam itself into the kernel. And the convoluted process to do it failed (it's packaged for Ubuntu but I was installing it on a slightly outdated Fedora system).
What, this part is only needed for secure boot? I'm not sec... oh. So go back to the UEFI settings, turn secure boot off, problem solved. I usually also turn off SELinux right after install.
So I'm an old greybeard who likes to have full control. Less secure. But at least I get the choice. Hopefully I continue to do so. The notion of not being able to access online banking services or other things that require account login, without running on a "fully attested" system does worry me.
Secure Boot only extends the chain of trust from your firmware down the first UEFI binary it loads.
Currently SB is effectively useless because it will at best authenticate your kernel but the initrd and subsequent userspace (including programs that run as root) are unverified and can be replaced by malicious alternatives.
Secure Boot as it stands right now in the Linux world is effectively an annoyance that’s only there as a shortcut to get distros to boot on systems that trust Microsoft’s keys but otherwise offer no actual security.
It however doesn’t have to be this way, and I welcome efforts to make Linux just as secure as proprietary OSes who actually have full code signature verification all the way down to userspace.
A basic setup to make use of secure boot is SB+TPM+LUKS. Unfortunately I don't know of any distro that offers this in a particularly robust way.
Code signature verification is an interesting idea, but I'm not sure how it could be achieved. Have distro maintainers sign the code?
Exciting!
It sounds like you want to achieve system transparency, but I don't see any clear mention of reproducible builds or transparency logs anywhere.
I have followed systemd's efforts into Secure Boot and TPM use with great interest. It has become increasingly clear that you are heading in a very similar direction to these projects:
- Hal Finney's transparent server
- Keylime
- System Transparency
- Project Oak
- Apple Private Cloud Compute
- Moxie's Confer.to
I still remember Jason introducing me to Lennart at FOSDEM in 2020, and we had a short conversation about System Transparency.
I'd love to meet up at FOSDEM. Email me at fredrik@mullvad.net.
Edit: Here we are six years later, and I'm pretty sure we'll eventually replace a lot of things we built with things that the systemd community has now built. On a related note, I think you should consider using Sigsum as your transparency log. :)
Edit2: For anyone interested, here's a recent lightning talk I did that explains the concept that all project above are striving towards, and likely Amutable as well: https://www.youtube.com/watch?v=Lo0gxBWwwQE
Hi, I'm David, founding product lead.
Our entire team will be at FOSDEM, and we'd be thrilled to meet more of the Mullvad team. Protecting systems like yours is core to us. We want to understand how we put the right roots of trust and observability into your hands.
Edit: I've reached out privately by email for next steps, as you requested.
Hi David. Great! I actually wasn't planning on going due to other things, but this is worth re-arranging my schedule a bit. See you later this week. Please email me your contact details.
As I mentioned above, we've followed systemd's development in recent years with great interest, as well as that of some other projects. When I started(*) the System Transparency project it was very much a research project.
Today, almost seven years later, I think there's a great opportunity for us to reduce our maintenance burden by re-architecting on top of systemd, and some other things. That way we can focus on other things. There's still a lot of work to do on standardizing transparency building blocks, the witness ecosystem(**), and building an authentication mechanism for system transparency that weaves it all together.
I'm more than happy to share my notes with you. Best case you build exactly what we want. Then we don't have to do it. :)
*: https://mullvad.net/en/blog/system-transparency-future
**: https://witness-network.org
I'm super far from an expert on this, but it NEEDS reproducible builds, right? You need to start from a known good, trusted state - otherwise you cannot trust any new system states. You also need it for updates.
Well, it comes down to what trust assumptions you're OK with. Reproducible reduces trust in the build environment, but you still need to ensure authenticity of the source somehow. Verified boot, measured boot, repro builds, local/remote attestation, and transparency logging provide different things. Combined they form the possibility of a sort of authentication mechanism between a server and client. However, all of the concepts are useful by themselves.
>Amutable is based out of Berlin, Germany.
Probably obvious from the surnames but this is the first time I've seen a EU company pop up on Hacker News that could be mistaken for a Californian company. Nice to see that ambition.
I understand systemd is controversial, that can be debated endlessly but the executive team and engineering team look very competitive. Will be interesting to see where this goes.
Lennart will be involved with at least three events at FOSDEM on the coming weekend. The talks seem unrelated at first glance but maybe there will be an opportunity to learn more about his new endeavor.
https://fosdem.org/2026/schedule/speaker/lennart_poettering/
Also see http://amutable.com/events which lists a talk at Open Confidential Computing Conference (Berlin, March)
No. Esp with LP’s track record in systemd.
See: “it’s just an init system”where it’s now also a resolver, log system, etc.
I can buy good intentions, but this opens up too much possibility for not-so-good-intended consequences. Deliberate or emergent.
systemd solved/improved a bunch of things for linux, but now the plan seems to be to replace package management with image based whole dist a/b swaps. and to have signed unified kernel images.
this basically will remove or significantly encumber user control over their system, such that any modification will make you loose your "signed" status and ... boom! goodbye accessing the internet without an id
pottering recently works for Microsoft, they want to turn linux into an appliance just like windows, no longer a general purpose os. the transition is still far from over on windows, but look at android and how the google play services dependency/choke-hold is
im sure ill get many down votes, but despite some hyperbole this is the trajectory
Good thing, without the power coming from RedHat money, the capacity of ruining the Linux ecosystem will finally be reduced!
What will they be reinventing from scratch for no reason?
The first steps look similar to secure boot with TPM.
It starts from there, then systemd takes over and carries the flag forward.
See the "features" list from systemd 257/258 [0].
[0]: https://0pointer.net/blog/
So I imagine Lennart Poettering has left Microsoft.
Looking forward to never using any of this, quite frankly; and hoping it remains optional for the kernel.
If there’s a path to profitability, great for them, and for me too; because it means it won’t be available at no charge.
Can someone smarter than myself describe immutability versus atomicity in regards to current operating systems on the market?
Immutability means you can't touch or change some parts of the system without great effort (e.g. macOS SIP).
Atomicity means you can track every change, and every change is so small that it affects only one thing and can be traced, replayed or rolled back. Like it's going from A to B and being able to return back to A (or going to B again) in a determinate manner.
So LP is or has left Microsoft ?
>We are building cryptographically verifiable integrity into Linux systems
I wonder what that means ? It could be a good thing, but I tend to think it could be a privacy nightmare depending on who controls the keys.
Yes, I have.
Verifiable to who? Some remote third party that isn't me? The hell would I want that?
Just an assumption here, but the project appears to be about the methodology to verify the install. Who holds the keys is an entirely different matter.
Werner Von Braun only built the rockets; he didn't aim them, nor did he care where they landed.
(London. On some of my relatives.)
The events includes a conference title "Remote Attestation of Imutable Operating Systems built on systemd", which is a bit of a clue.
I'm sure this company is more focused on the enterprise angle, but I wonder if the buildout of support for remote attestation could eventually resolve the Linux gaming vs. anti-cheat stalemate. At least for those willing to use a "blessed" distro.
Road to hell is paved with good intentions.
Somebody will use it and eventually force it if it exists and I don't think gaming especially those requiring anti-cheat is worth that risk.
If that means linux will not be able to overtake window's market share, that's ok. At-least the year of the linux memes will still be funny.
Only by creating a new stalemate between essential liberty and a little temporary security — anticheat doesn't protect you from DMA cheating.
I might be behind on the latest counter-counter-counter-measures, but I know some of the leading AC solutions are already using IOMMU to wedge a firewall between passive DMA sniffers and the game processes memory.
e.g. https://support.faceit.com/hc/en-us/articles/19590307650588-...
I sincerely hope not.
rust-vmm-based environment that verifies/authenticates an image before running ? Immutable VM (no FS, root dropper after setting up network, no or curated device), 'micro'-vm based on systemd ? vmm captures running kernel code/memory mapping before handing off to userland, checks periodically it hasn't changed ? Anything else on the state of the art of immutable/integrity-checking of VMs?
Sounds like kernel mode DRM or some similarly unwanted bullshit.
It's probably built on systemd's Secure Boot + immutability support.
As said above, it's about who controls the keys. It's either building your own castle or having to live with the Ultimate TiVo.
We'll see.
We all know who controls the keys. It's the first party who puts their hands on the device.
Doesn't have to be. While I'm not a fan of systemd (my comment history is there), I want to start from a neutral PoV, and see what it does.
I have my reservations, ideas, and what it's supposed to do, but this is not a place to make speculations and to break spirits.
I'll put my criticism out politely when it's time.
Just to make it clear - on Android you don't have the keys. Even with avb_custom_key you can't modify many partitions.
None of the consumer mobile devices give you all the keys. There are many reasons for that, but 99.9% of them are monetary reasons.
> Sounds like kernel mode DRM or some similarly unwanted bullshit.
Look, I hate systemd just as much as the next guy - but how are you getting "DRM" out of this?
Remote attestation is literally a form of DRM
There are genuine positive applications for remote attestation. E.g., if you maintain a set of servers, you can verify that it runs the software it should be running (the software is not compromised). Or if you are running something similar to Apple's Private Compute Cloud to run models, users can verify that it is running the privacy-preserving image that it is claiming to be running.
There are also bad forms of remote attestation (like Google's variant that helps them let banks block you if you are running an alt-os). Those suck and should be rejected.
Edit: bri3d described what I mean better here: https://news.ycombinator.com/item?id=46785123
I agree that DRM feels good when you're the one controlling it.
As the immediate responder to this comment, I claim to be the next guy. I love systemd.
"cryptographically verifiable integrity" is a euphemism for tivoization/Treacherous Computing. See, e.g., https://www.gnu.org/philosophy/can-you-trust.en.html
Secure boot and attestation both generally require a form of DRM. It’s a boon for security, but also for control.
I don't mind SystemD.
Hacker News has recently been dominated by conspiracy theorists who believe that all applications of cryptography are evil attempts by shadowy corporate overlords to dominate their use of computing.
No, it's not "all applications of cryptography". It's only remote attestation.
Buddy, if I want encryption of my own I've got secure boot, LUKS, GPG, etc. With all of those, why would I need or even want remote attestation? The purpose of that is to assure corporations that their code is running on my computer without me being able to modify it. It's for DRM.
I am fairly confident that this company is going to assure corporations that their own code is running on their own computers (ie - to secure datacenter workloads), to allow _you_ (or auditors) to assure that only _your_ asserted code is also running on their rented computers (to secure cloud workloads), or to assure that the code running on _their_ computers is what they say it is, which is actually pretty cool since it lets you use Somebody Else's Computer with some assurance that they aren't spying on you (see: Apple Private Cloud Compute). Maybe they will also try to use this to assert "deep" embedded devices which already lock the user out, although even this seems less likely given that these devices frequently already have such systems in place.
IMO it's pretty clear that this is a server play because the only place where Linux has enough of a foothold to make client / end-user attestation financially interesting is Android, where it already exists. And to me the server play actually gives me more capabilities than I had: it lets me run my code on cloud provided machines and/or use cloud services with some level of assurance that the provider hasn't backdoored me and my systems haven't been compromised.
How can you be "pretty sure" they're going to develop precisely the technology needed to implement DRM but also will never use or allow it to be used by anybody but the lawful owners of the hardware? You can't.
Are there VCs who participated in funding this or are you self funded?
The typical HN rage-posting about DRM aside, there's no reason that remote attestation can't be used in the opposite direction: to assert that a server is running only the exact code stack it claims to be, avoiding backdoors. This can even be used with fully open-source software, creating an opportunity for OSS cloud-hosted services which can guarantee that the OSS and the build running on the server match. This is a really cool opportunity for privacy advocates if leveraged correctly - the idea could be used to build something like Apple's Private Cloud Compute but even more open.
Like evil maid attacks, this is a vanishingly rare scenario brought out to try to justify technology that will overwhelmingly be used to restrict computing freedom.
intel have had a couple of goes at this
and each time the doors have been blasted wide off by huge security vulnerabilities
the attack surface is simply too large when people can execute their own code nearby
You're absolutely right, but considering Windows requirements drive the PC spec, this capability can be used to force Linux distributions in bad ways.
So, some of the people doing "typical HN rage-posting about DRM" are also absolutely right.
The capabilities locking down macOS and iOS and related hardware also can be used for good, but they are not used for that.
> but considering Windows requirements drive the PC spec, this capability can be used to force Linux distributions in bad ways
What do you mean by this?
Is the concern that systemd is suddenly going to require that users enable some kind of attestation functionality? That making attestation possible or easier is going to cause third parties to start requiring it for client machines running Linux? This doesn't even really seem to be a goal; there's not really money to be made there.
As far as I can tell the sales pitch here is literally "we make it so you can assure the machines running in your datacenter are doing what they say they are," which seems pretty nice to me, and the perversions of this to erode user rights are either just as likely as they ever were or incredibly strange edge cases.
Microsoft has a "minimum set of requirements" document about "Designed for Windows" PCs. You can't sell a machine with Windows or tell it's Windows compatible without complying with that checklist.
So, every PC sold to consumers is sanctioned by Microsoft. This list contains Secure Boot and TPM based requirements, too.
If Microsoft decides to eliminate enrollment of user keys and Secure Boot toggle, they can revoke current signing keys for "shims" and force Linux distributions to go full immutable to "sign" their bootloaders so they can boot. As said above, it's not something Amutable can control, but enable by proxy and by accident.
Look, I work in a datacenter, with a sizeable fleet. Being able to verify that fleet is desirable for some kinds of operations, I understand that. On the other hand, like every double edged sword, this can cut in both ways.
I just want to highlight that, that's all.
I don't see how this relates in any way to Amutable and it has been a "concern" for 20+ years (which has never come to pass). How do you think this relates at all?
Before this point in time, Linux never supported being an immutable image. Neither filesystems, nor the mechanism to lock it down was there. The best you could do was, TiVoization, but that would be too obvious and won't fly.
Now we have immutable distributions (SuSE, Fedora, NixOS). We have the infrastructure for attestation (systemd's UKI, image based boot, and other immutability features), TPMs and controversially uutils (Which is MIT licensed and has the stated goal to replace all GNU userspace).
You can build an immutable and adversarial userspace where you don't have to share the source, and require every boot and application call to attest. The theoretical thickness of the wall is both much greater and this theoretical state is much easier to achieve.
20 years ago the only barrier was booting. After that everything was free. Now it's possible to boot into a prison where your every ls and cd command can be attested.
Oh, Rust is memory safe. Good luck finding holes.
> Before this point in time, Linux never supported being an immutable image.
What? As just one example, dm-verity was merged into the mainline kernel 13 years ago. I built immutable, verified Linux systems at least ten years ago, and it was considered old hat by the time I got there.
> The best you could do was, TiVoization, but that would be too obvious and won't fly.
What does this even mean? "TiVoization" is the slang for "you get a device that runs Linux, you get the GPL sources, but you can't flash your own image on the device because you don't own the keys." This is the exact same problem then as it was now and just as "obvious?"
I understand the fears that come from client attestation (certainly, the way it has been used on Android has been majorly detrimental to non-Google ROMs), but, to the Android point, the groundwork has always been there.
I'd be very annoyed if someone showed up and said "we're making a Linux-based browser attestation system that your bank is going to partner on," but nobody has even gone this direction on Windows yet.
> Oh, Rust is memory safe. Good luck finding holes.
I break secure boot systems for a living and I'd say _maybe_ half of the bugs I find relate to memory safety in a way Rust would fix. A lot of systems already use tools which provide very similar safety guarantees to Rust for single threaded code. Systems are definitely getting more secure and I do worry about impenetrable fortresses appearing in the near future, but making this argument kind of undermines credibility in this space IMO.
Thank you Lennart, I hope you will be now sufficiently busy to not contribute anything into Linux anymore