Speaking to the "uncensored model" angle: there's little reason to distribute abliterated weights anyway. Instead of orthogonalising the weights that write back to the residual stream, you can just orthogonalise the activations themselves. It's equivalent.
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.
This is the original description of abliteration and it's quite approachable and interesting to read: https://arxiv.org/abs/2406.11717
There's an empirical observation that models often have a single direction in their activation space for "hmm no I shouldn't do this". It forms naturally during pre-training, and is then surfaced during post-training to make the model refuse to engage in certain behaviour.
With a little bit of linear algebra you can zap that direction from the model's activations, and it stops refusing to do things. You can also do the opposite: magnify that direction, and the model refuses to do absolutely anything.
Instead of editing the weights so they don't create the refusal signal, just let them do whatever, then delete the refusal signal itself. You don't want to edit quantised weights because it causes a loss of precision that can be pretty bad.
Torrents should really be the preferred method for distributing AI model weights. Why rely on a single point of failure like Hugging Face? BitTorrent was made for exactly this.
In my experience public torrents often die as they grow older. It doesn't help that BitTorrent V1 makes long term seeding annoying, and BitTorrent V2 is almost never used.
I never understood this, is there anything that makes it difficult for the original uploader, the one that supposedly offers the file directly, to offer a torrent instead for the same amount of time?
As far as perennity is concerned it seems strictly better.
> Distros are a bad use case for P2P anyway since you depend on upstream as soon as you start upgrading and installing packages.
This is true for any distribution method not just p2p. You can even download a nightly through torrents so what does it matter how the data is transferred if it’s always going to require `apt update`?
You're absolutely right. The problem is convenience. Torrent doesn't integrate as nicely as curl, wget, apt-get, npm, you name it or even in the browser. It doesn't have to be that way but it is currently.
Is there a fully in-browser torrent option that has the same UX as a regular file download in Firefox? Even better for others/the system would be to then keep sharing by default as long as the file's on your drive..
The biggest problem with BTv1 was the lack of per-file checksumming, and swarm merging (i.e. individual files have shared seeding pools across torrents). BTv2 specs the latter, but I think only BiglyBT actually implements it. Having both of those features from the get-go would've gone a LONG way to fixing the dead torrent problem.
Because history is path-dependent, as engineers keep learning over and over again. It doesn't matter whether Plan9 is theoretically superior to Linux - we're all on Linux and nobody's porting all the apps over.
ages ago I tried using IPFS to more or less accomplish this, I imagined it to act more like a weights/training data network fs that everyone would be able to participate in.
Once I was using Blizzard's downloader to install something (StarCraft, Diablo, I don't remember), and it was kinda slow. I disabled P2P downloads and speed skyrocketed, and I said "Huh, this was unexpected".
When P2P downloads disabled you could see the list of CDNs you're downloading from and mine had a single IP on that list. It looked familiar. Then it dawned on to me. It was the Akamai server which we were hosting in our system room, at 15 minutes of driving distance. After a chuckle, I went to get a cup of tea, because that was entertaining than the game itself.
Then of course, I dived into whatever I was installing that night.
Same here. Inspired by both, I used to delivery videos over torrent to in door machines since at that time, there's no CDNs (or it was difficult to get one).
Several companies tried this for distributing software.
It was very controversial. Users were angry that software companies were using their internet bandwidth to distribute their software. Made a lot of people angry.
Torrents always seemed like the more sensible way to distribute model weights.
Though I have been disappointed that most of these have been spurred on by the misleading claim that abliterated models were being taken down from HuggingFace because they removed an abliterated model. HF took one abliterated model down because the uploader was spamming people who requested access with sketchy requirements to pay for it.
Plus, there are a bunch of these types of sites, all of them have a couple of models and otherwise completely dead.
There's also the problem of catching malicious models that have been fine tuned to exfiltrate credentials. It would be nice to have means of checking hashes against the HF versions (or against other reputable sources). I'm guessing this is probably easy when just serving the same folder as what HF serves.
I’ve been wondering when this will come. The days are numbered for abliterated models to be published on HF I think. Why wouldn’t the government want a central control there?
Honestly, models are torrents will end any effort from the big AI labs to stop open models. No way to prevent weights from being shared, just like mobies. Genie is out of the bottle
If I search for 'uncensored' there are no torrents available. Uncensored models should be top priority, especially now that Nvidia owns HuggingFace and will enshittify the platform in accordance with upcoming US laws.
There are such results on huggingface tho. Also search for the keywords “abliteration” and “heretic”. Heretic is a tool used to abliterate, that is decensor, models
That was my my first search too. I know they are still listed on HF but every time I try and use one, the links are dead or the size is beyond my scope. Was hoping for a fresh batch. Ill check back.
great initiative, it's really weird seeing efficiencies get rediscovered in the LLM audience, because these efficiencies aren't even what I would consider to be old
Speaking to the "uncensored model" angle: there's little reason to distribute abliterated weights anyway. Instead of orthogonalising the weights that write back to the residual stream, you can just orthogonalise the activations themselves. It's equivalent.
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.
Can you explain this a bit to a non-expert?
I haven't wrapped my mind around this
This is the original description of abliteration and it's quite approachable and interesting to read: https://arxiv.org/abs/2406.11717
There's an empirical observation that models often have a single direction in their activation space for "hmm no I shouldn't do this". It forms naturally during pre-training, and is then surfaced during post-training to make the model refuse to engage in certain behaviour.
With a little bit of linear algebra you can zap that direction from the model's activations, and it stops refusing to do things. You can also do the opposite: magnify that direction, and the model refuses to do absolutely anything.
Instead of editing the weights so they don't create the refusal signal, just let them do whatever, then delete the refusal signal itself. You don't want to edit quantised weights because it causes a loss of precision that can be pretty bad.
Torrents should really be the preferred method for distributing AI model weights. Why rely on a single point of failure like Hugging Face? BitTorrent was made for exactly this.
In my experience public torrents often die as they grow older. It doesn't help that BitTorrent V1 makes long term seeding annoying, and BitTorrent V2 is almost never used.
A torrent with a webseed is strictly more resilient than a direct download link alone.
I never understood this, is there anything that makes it difficult for the original uploader, the one that supposedly offers the file directly, to offer a torrent instead for the same amount of time?
As far as perennity is concerned it seems strictly better.
Every change to the source is effectively a new torrent. This creates a ton of fragmentation as data is reorganized, remixed, reencoded, and so on.
You can see this with many Linux distros: there is no single Debian torrent that people seed for years because there's always a refreshed version.
Distros are a bad use case for P2P anyway since you depend on upstream as soon as you start upgrading and installing packages.
> Distros are a bad use case for P2P anyway since you depend on upstream as soon as you start upgrading and installing packages.
This is true for any distribution method not just p2p. You can even download a nightly through torrents so what does it matter how the data is transferred if it’s always going to require `apt update`?
If they're no longer using that model they may not be willing to continue using their storage for it.
How does this address the previous point? If they are not providing storage, then centralized or decentralized doesn’t make a difference.
Torrent/P2P can only add redundancy, so it’s impossible to have worse availability than a download link?
You're absolutely right. The problem is convenience. Torrent doesn't integrate as nicely as curl, wget, apt-get, npm, you name it or even in the browser. It doesn't have to be that way but it is currently. Is there a fully in-browser torrent option that has the same UX as a regular file download in Firefox? Even better for others/the system would be to then keep sharing by default as long as the file's on your drive..
> Is there a fully in-browser torrent option that has the same UX as a regular file download in Firefox?
Opera did back in the day.
The biggest problem with BTv1 was the lack of per-file checksumming, and swarm merging (i.e. individual files have shared seeding pools across torrents). BTv2 specs the latter, but I think only BiglyBT actually implements it. Having both of those features from the get-go would've gone a LONG way to fixing the dead torrent problem.
You only need one person/organization to commit to seeding. The majority of people do not want to seed at all without some sort of incentive.
If this site represents a coordinated datahoarding effort then there will be at least a few people who will seed indefinitely.
The last guy (kimdotcom) who was working on this (incentive for seeding) seems to be heading to the US: https://www.rnz.co.nz/news/science-and-technology/651123/cou...
It’s interesting he’s no longer getting any media attention any more.
I always wondered why v2 is never used... you can even search for files by their individual hash with it.
Because history is path-dependent, as engineers keep learning over and over again. It doesn't matter whether Plan9 is theoretically superior to Linux - we're all on Linux and nobody's porting all the apps over.
ages ago I tried using IPFS to more or less accomplish this, I imagined it to act more like a weights/training data network fs that everyone would be able to participate in.
Yeah, I thought they were used for this already. Surprised this is news, but also relieved.
IIRC Mistral used to do it, not sure if that's still the case
EDIT/ Yes they did, that no longer seems to be the case though
https://x.com/MistralAI/status/1833758285167722836
Steam & Blizzard (probably others) used to delivery games through torrent protocol in the past, before CDNs became cheaper.
When StarCraft 2 was lauched, the installer (before Battle.net installer crapware) had a complete graphical visualization of seeders & leechers.
Reference: https://warcraft.wiki.gg/wiki/Blizzard_Downloader
Oh, story time:
Once I was using Blizzard's downloader to install something (StarCraft, Diablo, I don't remember), and it was kinda slow. I disabled P2P downloads and speed skyrocketed, and I said "Huh, this was unexpected".
When P2P downloads disabled you could see the list of CDNs you're downloading from and mine had a single IP on that list. It looked familiar. Then it dawned on to me. It was the Akamai server which we were hosting in our system room, at 15 minutes of driving distance. After a chuckle, I went to get a cup of tea, because that was entertaining than the game itself.
Then of course, I dived into whatever I was installing that night.
I've worked at trading firms where the "reference data master" file is usually a big json or equivalent.
To get the file out to 100s or 1000s of machine they would often use private bittorent to distribute the file out.
As a private network, they have the option of multicast.
Same here. Inspired by both, I used to delivery videos over torrent to in door machines since at that time, there's no CDNs (or it was difficult to get one).
Several companies tried this for distributing software.
It was very controversial. Users were angry that software companies were using their internet bandwidth to distribute their software. Made a lot of people angry.
Arguably this should've been a thing since day 1 (and probably would've helped to prevent the buyout), but better late than never.
Hugging face seemed like buyout bait from day one.
Which makes it more surprising that something like this didn't exist. I'm relieved it does though.
I'd support this if I didn't have to post to xitter to claim my username.
Really odd approach.
Torrents always seemed like the more sensible way to distribute model weights.
Though I have been disappointed that most of these have been spurred on by the misleading claim that abliterated models were being taken down from HuggingFace because they removed an abliterated model. HF took one abliterated model down because the uploader was spamming people who requested access with sketchy requirements to pay for it.
Plus, there are a bunch of these types of sites, all of them have a couple of models and otherwise completely dead.
There's also the problem of catching malicious models that have been fine tuned to exfiltrate credentials. It would be nice to have means of checking hashes against the HF versions (or against other reputable sources). I'm guessing this is probably easy when just serving the same folder as what HF serves.
I’ve been wondering when this will come. The days are numbered for abliterated models to be published on HF I think. Why wouldn’t the government want a central control there?
That is pretty smart, torrent should have been used for more things, and this is a perfect usecase!
So begins the Merovingian and the Exiles..
Honestly, models are torrents will end any effort from the big AI labs to stop open models. No way to prevent weights from being shared, just like mobies. Genie is out of the bottle
If I search for 'uncensored' there are no torrents available. Uncensored models should be top priority, especially now that Nvidia owns HuggingFace and will enshittify the platform in accordance with upcoming US laws.
There are such results on huggingface tho. Also search for the keywords “abliteration” and “heretic”. Heretic is a tool used to abliterate, that is decensor, models
That was my my first search too. I know they are still listed on HF but every time I try and use one, the links are dead or the size is beyond my scope. Was hoping for a fresh batch. Ill check back.
great initiative, it's really weird seeing efficiencies get rediscovered in the LLM audience, because these efficiencies aren't even what I would consider to be old
but I guess they are
Now I want GPT-4.5