> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work.
Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
Another possible reason that occurred to me: what if you have VRAM but you're not using it all the time? For example, let's say you bought a GPU because you like to play video games. When you're not actively gaming, you probably don't need 16 GB of VRAM just to render the desktop. Might as well use it for something else, right?
Edit: Although, this is predicated on the system being able to release VRAM that is acting as swap when it's time to start a game. Can it do that?
What about backpressure, how does it handle requirements for VRAM allocation when VRAM is used for swap space?
With X11 it's not that bad (buffers are pre-allocated), but with Wayland allocations are a lot more dynamic, so running low on VRAM can easily crash the whole desktop. I just had a few of such crashes with Hyprland+llama-server+KVM switching between computers without freeing VRAM.
Remember how 16GBs used to be an enterprise level database mainframe?
Well, GPUs also have stupid amounts of compute on them. I have to imagine that there is some kind of database format that's useful with GPU compute attached.
Since the data is already in VRAM, the GPU can sort, join, or otherwise manipulate data as needed.
GPU-accelerated databases have a long history. I founded HeavyAI (previously MapD/OmniSci) in 2013, but there are or have been many other startups in this space, such as Voltron Data, Kinetica, Sqream, etc. And now you have major players like IBM, Starburst, and Microsoft (which just announced Fabric SQL on GPU today) working on their own GPU-accelerated systems. GPUs have a huge advantage in terms of compute, memory, and interconnect bandwidth over CPU, as long as you can keep them fed with data.
I believe within 2-3 years databases and data warehouses on GPU will be common. The widespread use of agents to query data will be a part of this, as there will be a need to run far more queries at lower latency than needed for the ETL and BI workloads of the past.
This HN comment and the linked post brought up a lot of good points. The main takeaway is that swap should primarily be considered a mechanism for equality of reclamation, not for emergency extra memory, where equality of reclamation means file-backed pages and anonymous pages are subject to similar criteria for being evicted from physical memory.
I used to have zero swap on my Linux desktop and this convinced me to add at least a small swap partition.
It really depends on what you run and how much RAM you have to do it in. I run some machines into swap just by running a couple browsers and some containers in the background on a 16GB laptop. I've also run a single light browser and essentially nothing else on 4GB and been fine:)
They already do that on windows and it kinda sucks. If you are targeting something like LMStudio or ComfyUI, both of those have superior methods to do exactly this.
Back when HDDs were all there was ramdisks were interesting, but SSDs pretty much killed most of that as they have massively increased IOPS over disks.
Hard drives that huge scare me as it would take days to backup all the data off them.
Wouldn't it be faster to swap to vram if you are sitting there with 8gigs of it unused than swapping to ssd and burning its write cycles, assuming you absolutely need swap
> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work.
Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
Another possible reason that occurred to me: what if you have VRAM but you're not using it all the time? For example, let's say you bought a GPU because you like to play video games. When you're not actively gaming, you probably don't need 16 GB of VRAM just to render the desktop. Might as well use it for something else, right?
Edit: Although, this is predicated on the system being able to release VRAM that is acting as swap when it's time to start a game. Can it do that?
What about backpressure, how does it handle requirements for VRAM allocation when VRAM is used for swap space?
With X11 it's not that bad (buffers are pre-allocated), but with Wayland allocations are a lot more dynamic, so running low on VRAM can easily crash the whole desktop. I just had a few of such crashes with Hyprland+llama-server+KVM switching between computers without freeing VRAM.
Given my dev machine has 32GB of RAM and 32GB of VRAM that sits mostly idle when I'm not running AI models, this is not that bad of an idea.
Nice idea, but something has gone very wrong here:
>Sequential throughput: ~1.3 GB/s
[on a RTX 3070 Laptop]
This RTX 3070 chip is on PCIe 4.0 x16 which should give 64GB/s. The 8GB of GDDR6 is 448GB/s.
Swapping to an NVMe drive would be twice as fast, but with higher latency.
Remember how 16GBs used to be an enterprise level database mainframe?
Well, GPUs also have stupid amounts of compute on them. I have to imagine that there is some kind of database format that's useful with GPU compute attached.
Since the data is already in VRAM, the GPU can sort, join, or otherwise manipulate data as needed.
GPU-accelerated databases have a long history. I founded HeavyAI (previously MapD/OmniSci) in 2013, but there are or have been many other startups in this space, such as Voltron Data, Kinetica, Sqream, etc. And now you have major players like IBM, Starburst, and Microsoft (which just announced Fabric SQL on GPU today) working on their own GPU-accelerated systems. GPUs have a huge advantage in terms of compute, memory, and interconnect bandwidth over CPU, as long as you can keep them fed with data.
I believe within 2-3 years databases and data warehouses on GPU will be common. The widespread use of agents to query data will be a part of this, as there will be a need to run far more queries at lower latency than needed for the ETL and BI workloads of the past.
Can we somehow make them work with 1 TB PCIes so we can churn through way more data?
oh god please don't create more demand for GPUs
Possibly LSM compaction.
Does anyone these days really use swap for anything than S4 suspend ?
https://news.ycombinator.com/item?id=40697318
This HN comment and the linked post brought up a lot of good points. The main takeaway is that swap should primarily be considered a mechanism for equality of reclamation, not for emergency extra memory, where equality of reclamation means file-backed pages and anonymous pages are subject to similar criteria for being evicted from physical memory.
I used to have zero swap on my Linux desktop and this convinced me to add at least a small swap partition.
It really depends on what you run and how much RAM you have to do it in. I run some machines into swap just by running a couple browsers and some containers in the background on a 16GB laptop. I've also run a single light browser and essentially nothing else on 4GB and been fine:)
You want to waste VRAM, in this economy?
It's 1GiB. What could it cost, $10?
can't stand the thought of people not understanding the above reply[0]
[0]: https://knowyourmeme.com/memes/its-one-banana-michael-what-c...
I'm more interested in the opposite. Nvidia linux drivers crash when you try to address more VRAM than you have. It'd be nice if they didn't.
They already do that on windows and it kinda sucks. If you are targeting something like LMStudio or ComfyUI, both of those have superior methods to do exactly this.
Finally a use for the expensive ram when it's not needed in workloads!
Now if it could be dynamically used and vacated on other GPU workloads?
Q: Why? A: Why not?
No software benchmarks? BAR for RAM is cool but I want to see how much it _actually_ beats pcie nvme
use your car for an anchor on a big boat!
I mean, if you aren’t using the car while using the boat and it won’t really damage the car… yes?
RAM disks have always fascinated me. In a different timeline every PC has a 100gb of RAM and 50TB HDDs are the norm.
Back when HDDs were all there was ramdisks were interesting, but SSDs pretty much killed most of that as they have massively increased IOPS over disks.
Hard drives that huge scare me as it would take days to backup all the data off them.
Nice
I mean, cool, but I’d rather not?
So don't. Not everything is for you.
Wouldn't it be faster to swap to vram if you are sitting there with 8gigs of it unused than swapping to ssd and burning its write cycles, assuming you absolutely need swap
So, erm, don't?