Sylve looks like a decent project with a promising future but this article really doesn't explain why they picked it over Proxmox at all. They explain a lot of things but I can't see the advantage over prox other than they wanted to use it.
I run Proxmox at home, but now that I have been drinking the NixOS koolaid over the past 2 years, all of my homelab problems suddenly look like Nix-shaped nails.
I have the same thing with proxmox especially after I realized how well it integrates with proxmox backup server. And I haven't even gotten into clustering yet. It really is a very solid product.
Without looking at the Sylve docs, I'll conjecture that it has deeper integration with ZFS. With a foundation on FreeBSD, there is a likelihood Sylve can support ZFS-on-root rollbacks better than hacking it into Proxmox. A rollback capability is why I'm looking for Proxmox alternatives. In the Linux world, Talos Linux and IncusOS provide A/B updates which achieve a similar rollback capability. With something based on FreeBSD, your "immutable" OS and all of it's data can be treated equally as ZFS datasets. There's also a higher risk that a Linux kernel update will break ZFS.
Regardless of the number of drives available, you gain an advantage when your file system can leverage snapshots to roll backwards or forwards. There are other Linux-native filesystems that can provide this capability too, but many admins prefer ZFS because the full range of capabilities is unparelleled.
Perhaps I'm missing your point, but proxmox+lxc on zfs storage works fine in proxmox? If just looks like any other storage in proxmox and on commandline you've got all the usual zfs tools
Nested virtualization can be very handy in both the lab and in production. In the lab, you can try out a new hosting platform by running one atop the other. IE: Proxmox on VMWare, Hyper-V on KVM. This lets you try things out without needing fresh bare metal hardware.
In prod, let's say you run workloads in Firecracker VMs. You have plenty of headroom on your existing hardware. Nested virtualization would allow you to set up Firecracker hosts on your existing hardware.
One example: when learning Proxmox itself. I was able to set up a multi-node cluster with more complicated networking than I was normally comfortable with and experiment with failures of all sorts (killing a node, disabling NICs, etc.) without needing more hardware or affecting my existing things.
Outside of learning and testing I am not sure of what uses there might be but I'm curious to know if there are.
I love FreeBSD but Linux just provides every feature under the sun when it comes to virtualization. Do you find any missing features on bhyve ? Is bhyve reliable ? I can't imagine its been tested as thoroughly as KVM ...
Bhyve is quite cool but no nested virt which means you cannot nest vm_enter/exit calls with EPT pages so you cannot virtualize within those guests. I found this crucial. For instance Qubes OS won't run in Bhyve by any means.
Anecdotally, Bhyve has worked in FreeBSD for a decade now. Eventually it got ported to Illumos because it was better than their implementation of QEMU.
If you are unsure of bhyve's abilities then why not test yourself? Speculation and guessing about stability or testing is useless without seeing if it works in your application.
> If you are unsure of bhyve's abilities then why not test yourself?
It is not possible to come to a conclusion about everything in the world yourself "from scratch". No one has the time to try out everything themselves. Some filteration process needs to be applied to prevent wasting your finite time.
That is why you ask for recommendations of hotels, restaurants, travel destinations, good computer brands, software and so on from friends, relatives or other trusted parties/groups. This does not mean your don't form your opinions. You use the opinions of others as a sort of bootstrap or prior which you can always refine.
HN is actually the perfect place to ask for opinions. Someone just said bhyve does not support nested virtualization (useful input !). Someone else might chime in and say they have run bhyve for a long time and they trust it (and so on...)
Sylve looks like a decent project with a promising future but this article really doesn't explain why they picked it over Proxmox at all. They explain a lot of things but I can't see the advantage over prox other than they wanted to use it.
Sometimes unification can be an advantage.
I run Proxmox at home, but now that I have been drinking the NixOS koolaid over the past 2 years, all of my homelab problems suddenly look like Nix-shaped nails.
Same. Here's how I scratch the NixOS itch on Proxmox and/or libvirt[1]. One interface for both targets.
[1] https://github.com/EnigmaCurry/nixos-vm-template
That feature list looks really good. It would actually be really nice to standardize the guest operating systems in such a way.
I actually have a few hosts that only run docker. I might be able to test with those.
I have the same thing with proxmox especially after I realized how well it integrates with proxmox backup server. And I haven't even gotten into clustering yet. It really is a very solid product.
Indeed, Proxmox VE is an amazing product.
What does Sylve provide that proxmox doesn't?
Or better, how does it do it better than proxmox?
This isn't to say that proxmox is the best thing since sliced bread, I'm curious as to what makes sylve better, is it the API?
Without looking at the Sylve docs, I'll conjecture that it has deeper integration with ZFS. With a foundation on FreeBSD, there is a likelihood Sylve can support ZFS-on-root rollbacks better than hacking it into Proxmox. A rollback capability is why I'm looking for Proxmox alternatives. In the Linux world, Talos Linux and IncusOS provide A/B updates which achieve a similar rollback capability. With something based on FreeBSD, your "immutable" OS and all of it's data can be treated equally as ZFS datasets. There's also a higher risk that a Linux kernel update will break ZFS.
> Sylve can support ZFS-on-root rollbacks better than hacking it into Proxmo
Can you explain your use case when you absolutely can't provide a separate M.2 drive solely for the OS?
Regardless of the number of drives available, you gain an advantage when your file system can leverage snapshots to roll backwards or forwards. There are other Linux-native filesystems that can provide this capability too, but many admins prefer ZFS because the full range of capabilities is unparelleled.
Perhaps I'm missing your point, but proxmox+lxc on zfs storage works fine in proxmox? If just looks like any other storage in proxmox and on commandline you've got all the usual zfs tools
Bhyve doesn't feature nested virt though.
Honestly asking, in which cases nested virtualization is useful?
Nested virtualization can be very handy in both the lab and in production. In the lab, you can try out a new hosting platform by running one atop the other. IE: Proxmox on VMWare, Hyper-V on KVM. This lets you try things out without needing fresh bare metal hardware.
In prod, let's say you run workloads in Firecracker VMs. You have plenty of headroom on your existing hardware. Nested virtualization would allow you to set up Firecracker hosts on your existing hardware.
One example: when learning Proxmox itself. I was able to set up a multi-node cluster with more complicated networking than I was normally comfortable with and experiment with failures of all sorts (killing a node, disabling NICs, etc.) without needing more hardware or affecting my existing things.
Outside of learning and testing I am not sure of what uses there might be but I'm curious to know if there are.
I love FreeBSD but Linux just provides every feature under the sun when it comes to virtualization. Do you find any missing features on bhyve ? Is bhyve reliable ? I can't imagine its been tested as thoroughly as KVM ...
Bhyve is quite cool but no nested virt which means you cannot nest vm_enter/exit calls with EPT pages so you cannot virtualize within those guests. I found this crucial. For instance Qubes OS won't run in Bhyve by any means.
Or Windows VBS, if you need to run full-featured Windows VMs.
Do they have any near-future plans to resolve that?
Anecdotally, Bhyve has worked in FreeBSD for a decade now. Eventually it got ported to Illumos because it was better than their implementation of QEMU.
If you are unsure of bhyve's abilities then why not test yourself? Speculation and guessing about stability or testing is useless without seeing if it works in your application.
> If you are unsure of bhyve's abilities then why not test yourself?
It is not possible to come to a conclusion about everything in the world yourself "from scratch". No one has the time to try out everything themselves. Some filteration process needs to be applied to prevent wasting your finite time.
That is why you ask for recommendations of hotels, restaurants, travel destinations, good computer brands, software and so on from friends, relatives or other trusted parties/groups. This does not mean your don't form your opinions. You use the opinions of others as a sort of bootstrap or prior which you can always refine.
HN is actually the perfect place to ask for opinions. Someone just said bhyve does not support nested virtualization (useful input !). Someone else might chime in and say they have run bhyve for a long time and they trust it (and so on...)
So I can't agree with your viewpoint.