Cool article! I'm working on a tangentially related issue requiring microVMs inside isolated infrastructure environments. Latency isn't really my main priority, but I am always tempted by any option to minimize attack surface. I wonder what it would take to replace the host block mount in this configuration with `vsock` for all communications between the host and guest microVM? Then you could avoid any files being mounted on the host at all while still enabling, e.g., one-way egress to a pre-signed S3 URL via a private VPC endpoint. Very cool article!
> I remember seeing a linux firewall/gateway set up to run with just the kernel, without any userspace at all. Completely unhackable.
Do you remember any details that would let me search for it? Because that does sound cool, and even maybe useful; the thought has certainly crossed my mind that a router or VPN box doesn't really get a lot of use out of userspace... Although maybe it's worth keeping for control/configuration/debugging.
> To print some text or run a simple program, I belive DOS without a memory manager would be even faster.
Or just make your code boot directly. It's not hard to make a .efi, or use
https://github.com/jart/cosmopolitan
to make a binary that runs in many places including bare metal.
IIRC, it ran a script as init process that set up the network cards, set up iptables, etc. and then just exited. Kernel would panic (the "init was killed" panic), but the network would still be functional. Automatic reboot on panic was disabled.
To reconfigure, the admin would simply reset it and start the system with "init=/something/else" as kernel parameter that booted to a normal userspace.
> I... fail to see the point of running just one process.
It makes sense if you got some legacy piece of hardware that has extremely limited resources, both in terms of RAM and storage. Write your code in Go and you don't even need libc any more.
Wouldn't this be useful for embedded types of applications where you have a very specific task you want to do and you want to do it now.... like that firewall example?
Do do something actually useful, the program would have to access some data: network, disk, some sensors, etc. Network alone means scanning PCIe for the network card and configuring it, disk access needs controller also on PCIe, then scanning the ports for the drive, reading partition table, mounting the partition, etc.
All that takes a lot more than 1s. The speedup might not even be significant compared to a kernel optimized for that system (all modules built-in, nothing redundant), but full-featured, plus busybox or sysvinit alone.
You may also want to build and run busybox for your tiny userspace.
Other things you may want to experiment with is gen_init_cpio.c from linux kernel tree. It makes creating initramfs file structure easier from scripts.
And finally if sys/isolinux is also fun to use for minimal boot images.
Cool article! I'm working on a tangentially related issue requiring microVMs inside isolated infrastructure environments. Latency isn't really my main priority, but I am always tempted by any option to minimize attack surface. I wonder what it would take to replace the host block mount in this configuration with `vsock` for all communications between the host and guest microVM? Then you could avoid any files being mounted on the host at all while still enabling, e.g., one-way egress to a pre-signed S3 URL via a private VPC endpoint. Very cool article!
> I found “Building a tiny Linux from scratch” which does most of what I do here but in Rust and a year ago
Linux from scratch seems to still be doing fine at: https://www.linuxfromscratch.org. It's going on 27 years now.
> yeah, I know, proper C code needs to be scattered with return value checks and sensible reports of errno. I’ve left these out for clarity.
Somewhere, a LLM is trained on this code as we speak :)
But anyway, it's great that people are still interested in learning this stuff for fun.
It's be cool to dual boot with a Linux that has a ~1s boot time, drops you into neovim and lets you save text files to a shared partition.
I do like this as an exploration.
It's possible to boot a VM noticeably faster still, though I'm unclear on whether any of that applies to hardware:
https://jvns.ca/blog/2021/01/23/firecracker--start-a-vm-in-l...
I... fail to see the point of running just one process.
If it's just a PoC, then:
1) I remember seeing a linux firewall/gateway set up to run with just the kernel, without any userspace at all. Completely unhackable.
2) To print some text or run a simple program, I belive DOS without a memory manager would be even faster.
3) It takes 1s to boot linux, but an ordinary PC takes 10s to get to that linux. Even U-boot on ARM takes some seconds to load a kernel.
BTW, if anyone knows any current platform that can XiP a linux kernel, please share.
> I remember seeing a linux firewall/gateway set up to run with just the kernel, without any userspace at all. Completely unhackable.
Do you remember any details that would let me search for it? Because that does sound cool, and even maybe useful; the thought has certainly crossed my mind that a router or VPN box doesn't really get a lot of use out of userspace... Although maybe it's worth keeping for control/configuration/debugging.
> To print some text or run a simple program, I belive DOS without a memory manager would be even faster.
Or just make your code boot directly. It's not hard to make a .efi, or use https://github.com/jart/cosmopolitan to make a binary that runs in many places including bare metal.
IIRC, it ran a script as init process that set up the network cards, set up iptables, etc. and then just exited. Kernel would panic (the "init was killed" panic), but the network would still be functional. Automatic reboot on panic was disabled.
To reconfigure, the admin would simply reset it and start the system with "init=/something/else" as kernel parameter that booted to a normal userspace.
Oh, clever; I didn't know you could make Linux panic but keep running.
> I... fail to see the point of running just one process.
It makes sense if you got some legacy piece of hardware that has extremely limited resources, both in terms of RAM and storage. Write your code in Go and you don't even need libc any more.
Wouldn't this be useful for embedded types of applications where you have a very specific task you want to do and you want to do it now.... like that firewall example?
Yeah, but there's a problem.
Do do something actually useful, the program would have to access some data: network, disk, some sensors, etc. Network alone means scanning PCIe for the network card and configuring it, disk access needs controller also on PCIe, then scanning the ports for the drive, reading partition table, mounting the partition, etc.
All that takes a lot more than 1s. The speedup might not even be significant compared to a kernel optimized for that system (all modules built-in, nothing redundant), but full-featured, plus busybox or sysvinit alone.
Embedded devices or other SoC situations, certain limited scope situations where docker is undesirable/unnecessary, etc.
"Butt Naked Linux" is how I read it.
I know it's off topic. I accept my downvotes.
I could end up stealing that for my own tinkering distro that never ends up releasing because it's awful.
You may also want to build and run busybox for your tiny userspace.
Other things you may want to experiment with is gen_init_cpio.c from linux kernel tree. It makes creating initramfs file structure easier from scripts.
And finally if sys/isolinux is also fun to use for minimal boot images.