Built-in Litestream replication and automatic restore is a new feature I added in OpenRun (a self-hosted deployment platform for web apps). Apps can use SQLite with zero code changes while OpenRun handles backup/recovery. This works on a single node with Docker/Podman and also on Kubernetes.
I use the same stack to run my apps! sqlite as Database backen on ephemeral pods. On initialisation, the pod restores the current litestream backup from object storage. No persistent volumes needed. No issues so far.
Without persistent volumes, every deployment of the app may require the SQLite data to be restored from S3, which can take time if database size is large. With PVC, regular app deployments use existing database, only a disaster recovery scenario restores from S3.
Yes, it feels magical to just recreate your namespace or even create a new K8s cluster and see all your apps (from OpenRun metadata) and app data restored automatically.
The ephemeral-pod-plus-restore-on-init pattern works right up until it doesn't, and the failure is quiet, so "no issues so far" is unfortunately consistent with both a correct setup and one that is going to bite.
Litestream assumes a single writer. It replicates one SQLite file's WAL to object storage; it is not multi-master. If you ever end up with two pods writing, you get two databases diverging and replicating over the top of each other, and nothing errors. You find out when you notice writes going missing.
The two ways people get there without meaning to:
- replicas gets bumped above 1, sometimes by an HPA nobody remembered was on.
- RollingUpdate. It briefly runs old and new pods together, which is enough. Recreate avoids it at the cost of a restart gap.
The other thing worth knowing is that replication is asynchronous with a default sync interval of one second, so a hard pod kill can lose whatever was written since the last sync. Fine for a lot of workloads. Not fine if the thing has quietly become a durable transaction log for something that matters.
LiteFS was the answer for genuinely multi-node, though I would check its current maintenance status before building on it.
On Kubernetes, for apps using SQLite, maxReplicas is set to 1 and OpenRun uses the Recreate update strategy. This ensures at most one pod is active (requests are queued until the new pod is healthy). There should never be more than one pod accessing the SQLite database.
Built-in Litestream replication and automatic restore is a new feature I added in OpenRun (a self-hosted deployment platform for web apps). Apps can use SQLite with zero code changes while OpenRun handles backup/recovery. This works on a single node with Docker/Podman and also on Kubernetes.
I use the same stack to run my apps! sqlite as Database backen on ephemeral pods. On initialisation, the pod restores the current litestream backup from object storage. No persistent volumes needed. No issues so far.
Without persistent volumes, every deployment of the app may require the SQLite data to be restored from S3, which can take time if database size is large. With PVC, regular app deployments use existing database, only a disaster recovery scenario restores from S3.
Yes, it feels magical to just recreate your namespace or even create a new K8s cluster and see all your apps (from OpenRun metadata) and app data restored automatically.
I have no use case where I store a TB of data in a sqlite let alone few GB. Currently this checks out and restore is few seconds.
I'm going to pass.
Do you have a particular issue with... the company that wrote this post, their choice of DB, decisions made in Litestream, SQLite, etc.?
Not a company :-) OpenRun has been a passion project I have been building for last three years, trying to make declarative deployments easier
The ephemeral-pod-plus-restore-on-init pattern works right up until it doesn't, and the failure is quiet, so "no issues so far" is unfortunately consistent with both a correct setup and one that is going to bite.
Litestream assumes a single writer. It replicates one SQLite file's WAL to object storage; it is not multi-master. If you ever end up with two pods writing, you get two databases diverging and replicating over the top of each other, and nothing errors. You find out when you notice writes going missing.
The two ways people get there without meaning to:
- replicas gets bumped above 1, sometimes by an HPA nobody remembered was on. - RollingUpdate. It briefly runs old and new pods together, which is enough. Recreate avoids it at the cost of a restart gap.
The other thing worth knowing is that replication is asynchronous with a default sync interval of one second, so a hard pod kill can lose whatever was written since the last sync. Fine for a lot of workloads. Not fine if the thing has quietly become a durable transaction log for something that matters.
LiteFS was the answer for genuinely multi-node, though I would check its current maintenance status before building on it.
On Kubernetes, for apps using SQLite, maxReplicas is set to 1 and OpenRun uses the Recreate update strategy. This ensures at most one pod is active (requests are queued until the new pod is healthy). There should never be more than one pod accessing the SQLite database.