There's a hybrid approach of C -> WASM -> C compilation, which ends up controlling every OS interaction and sandboxing memory access like WASM, while technically remaining C code:
WASM sandboxes don't do much to guarantee the soundness of your program. It can hose your memory all it wants, it can just only do so within the confines of the sandbox.
Using a sandbox also limits what you can do with a system. With stuff like SECCOMP you have to methodically define policies for all its interactions. Like you're dealing with two systems. It's very bureaucratic and the reason we do it, is because we don't trust our programs to behave.
With Fil-C you get a different approach. The language and runtime offer a stronger level of assurance your program can only behave, so you can trust it more to have unfettered access to the actual system. You also have the choice to use Fil-C with a sandbox like SECCOMP as described in the blog post, since your Fil-C binaries are just normal executables that can access powerful Linux APIs like prctl. It took Linux twenty years to invent that interface, so you'll probably have to wait ten years to get something comparable from WASI.
> It can hose your memory all it wants, it can just only do so within the confines of the sandbox.
True, although as I understand it the WASI component model at least allows multiple fine-grained sandboxes, so it's somewhere in-between per-object capabilities and one big sandbox for your entire program. I haven't actually used it yet so I might be wrong about that.
> so you'll probably have to wait ten years to get something comparable from WASI
I think for many WASI use cases the capability control would be done by the host program itself, so you don't need OS-level support for it. E.g. with Wasmtime I do
WASI is basically CORBA, and DCOM, PDO for newer generations.
Or if you prefer the bytecode based evolution of them, RMI and .NET Remoting.
I don't see it going that far.
The WebAssembly development experience on the browser mostly still sucks, especially the debugging part, and on the server it is another yet another bytecode.
Finally, there is hardly any benefit over OS processes, talking over JSON-RPC (aka how REST gets mostly used), GraphQL, gRPC, or plain traditional OS IPC.
WASM sacrifices guest security & performance in order to provide mediocre host sandboxing, though. It might be a useful tradeoff sometimes, but proper process-based sandboxing is so much stronger and lets the guest also have full security & performance.
How is process-based sandboxing stronger? Also the performance penalty is not only due to sandboxing (I doubt it's even mostly due to it). Likely more significant is the portability.
Because the guarantees themselves are stronger, process isolation is something we have decades of experience with, it goes wrong every now and then but those are rare instances whereas what amounts to application level isolation is much weaker in terms the guarantees it that it provides and the maturity level of the code. That suggests that if you base your isolation scheme on processes rather than 'just' sandboxing that you will come out ahead and even with all other things the same you'd have one more layer in your stack of swiss cheese slices. A VM would offer another layer of protection on top of that, one with yet stronger guarantees.
process-based sandboxing has hardware support and thus stronger defenses against things like spectre. So far every CPU on the market has only elected to address spectre as it relates to crossing ring or process boundaries. Nobody has added hardware spectre defenses for in-process sandboxing. Also, process-based sandboxing allows the guest to also have a full suite of security protections like ASLR. If you are doing sandboxing for defense in depth, reducing the security of what's inside the guest in turn reduces the security of your entire chain.
And I didn't say the performance penalty was because of sandboxing (although in the case of WASM there is cost as it's doing software enforcement of things that otherwise are "for free" in hardware), but just that WASM has a performance cost compared to native. If you are using WASM just for sandboxing, you still then pay a performance cost for portability you didn't need.
No because the host can present the same interface on every platform. I do think that WASI is waaay to much "let's just copy POSIX and screw other platforms", but it does work pretty well even on Windows.
You've named half of the weasel security technologies of the last three decades. The nice thing about SECCOMP BPF is it's so difficult to use that you get the comfort of knowing that only a very enlightened person could have written your security policy. Hell is being subjected to restrictions defined by people with less imagination than you.
Running ffmpeg compiled for wasm and watching as most codec selections lead to runtime crashes due to invalid memory accesses is fun. But, yeah, it’s runtime safety, so going to wasm as a middle step doesn’t do much.
> Running ffmpeg compiled for wasm and watching as most codec selections lead to runtime crashes due to invalid memory accesses is fun.
For all you know that’s a bug in the wasm port of the codec.
> it’s runtime safety
So is Fil-C
The problem with wasm is that an OOBA in one C allocation in the wasm guest can still give the attacker the power to clobber any memory in the guest. All that’s protected is the host. That’s enough to achieve weird execution.
Hence why I say that wasm is a sandbox. It’s not memory safety.
Finally reality is catching up with the WASM sales pitch against other bytecode formats introduced since 1958, regarding security and how great it is over anything else.
Warm was great because it was lightweight and easy to target from any language and create any custom interaction API with the host. That's becoming less true as they bolt on features no one needed (GC) and popularize standardized interfaces that contain the kitchen sink (WASI) but these things can still be treated as optional so it can still be used for much more flexible use cases than java or .net
There’s no guarantee the toolchains will support WASM “preview” forever and make the bloat optional, and even if they do you could still end up in an ecosystem where it would be unviable. At some point you’re probably better off just compiling to RISCV and using an emulator library instead.
Wasm now supports multiple modules and multiple linear memories per module, so it ought to be quite possible to compile C to Wasm in a way that enforces C's object access rules, much like CHERI if perhaps not Fil-C itself.
You wouldn't be able to get quite as fine-grained. One memory per object is probably horrifically slow. And I don't know about Fil-C, but CHERI at least allows capabilities (pointers with bounds) to overlap and subset each other. I.e. you could allocate an arena and get a capability for that, and then allocate an object inside that arena and get a smaller capability for that, and then get a pointer to a field in that object and get capability just for that field.
Fil-C has like one "linear memory" per object and each capability gives read/write access to the whole object.
But Fil-C has its compiler which does analysis passes for eliding bounds-checks where they are not needed, and I think it could theoretically do a better job at that than a WASM compiler with multi-memories, because C source code could contain more information.
Unlike WASM, but like CHERI, every pointer in memory is also tagged, and would lose its pointer status if overwritten by an integer, so it is still more memory-safe in that way.
One would probably just need to define WASM extensions that allow for such subsetting. Performance will probably be competitive with software implementations of CHERI (perhaps with varying levels of hardware acceleration down the road) which isn't that bad.
The author has a knack for generating buzz (and making technically interesting inventions) :)
I'm a little concerned that no one (besides the author?) has checked the implementation to see if reducing the attack surface in one area (memory security) might cause problems in other layers.
For example, Filip mentioned that some setuid programs can be compiled with it, but it also makes changes to ld.so. I pointed this out to the author on Twitter, as it could be problematic. Setuid applications need to be written super-defensively because they can be affected by envars, file descriptors (e.g. there could be funny logical bugs if fd=1/2 is closed for a set-uid app, and then it opens something, and starts using printf(), think about it:), rlimits, and signals. The custom modifications to ld.so likely don't account for this yet?
In other words, these are still teething problems with Fil-C, which will be reviewed and fixed over time. I just want to point out that using it for real-world "infrastructures" might be somewhat risky at this point. We need unix nerds to experiment with.
OTOH, it's probably a good idea to test your codebase with it (provided it compiles, of course) - this phase could uncover some interesting problems (assuming there aren't too many false positives).
Wishful thinking: Any possible chance that means you might make a Fil-C APE hybrid? It would neatly address the fact that Fil-C already needs all of its dependencies to also use Fil-C.
Yes, but instead of remarking solely on the fact that the author has a pretty good turnaround time for fixing bugs (I wished all open source projects were that fast) and listens to input belies the tone of your comment, which makes me come away with a negative view of the project, when in fact the evidence points to the opposite.
It's a 'damning with faint praise' thing and I'm not sure to what degree you are aware of it but I don't think it is a fair way to treat the author and the project. HN has enough of a habit of pissing on other people's accomplishments already. Critics have it easy, playwrights put in the hours.
I understand your point, and I have the utmost respect for the author who initiated, implemented, and published this project. It's a fantastic piece of work (I reviewed some part of it) that will very likely play an important role in the future - it's simply too good not to.
At the same time, however, the author seems to be operating on the principle: "If I don't make big claims, no one will notice." The statements about the actual security benefits should be independently verified -this hasn't happened yet, but it probably will, as the project is gaining increasing attention.
> "If I don't make big claims, no one will notice."
I am making big claims because there are big claims to be made.
> he statements about the actual security benefits should be independently verified -this hasn't happened yet
I don't know what this means. Folks other than me have independently verified my claims, just not exhaustively. No memory safe language runtime has been exhaustively verified, save maybe Spark. So you're either saying something that isn't true at all, or that could be said for any memory safe language runtime.
To clarify the position, my concern isn't that the project is bad - it's that security engineering is a two-front war. You have to add new protections (memory safety) without breaking existing contracts (like ld.so behavior).
When a project makes 'big claims' about safety, less technical users might interpret that as 'production ready'. My caution is caused by the fact that modifying the runtime is high-risk territory where regressions can introduce vulns that are distinct from the memory safety issues you are solving.
The goal is to prevent the regression in the first place. I'm looking forward to seeing how the verification matures and rooting for it.
> without breaking existing contracts (like ld.so behavior)
If you think that Fil-C regresses ld.so then get specific. Otherwise what you’re doing is spreading fear, uncertainty, and doubt for no good reason.
Fil-C has always honored the setuid behavior provided by ld.so. There was a bug - since fixed - that the Fil-C runtime called getenv instead of secure_getenv.
> When a project makes 'big claims' about safety, less technical users might interpret that as 'production ready'.
Fil-C is production ready and already has production users.
I would suggest you re-read your comment in a week or so to see if by then you are far enough away from writing it to see how others perceive it. If it wasn't your intention to be negative then maybe it is my non-native English capability that is the cause of this but even upon re-reading it that's how I perceive it.
- You start off with commenting that the author has a knack for self promotion and invention. My impression is that he's putting in a status report for a project that is underway.
- you follow this up with something that you can't possibly know and use that to put the project down, whilst at the same time positioning yourself as a higher grade authority because you are apparently able to see something that others do not, effectively doing that which you accuse the author of: self promotion.
- You then double down on this by showing that it was you who pointed out to the author that there was a bug in the software, which in the normal course of open source development is not usually enough to place yourself morally or technically above the authors.
- You then in your more or less official capacity of established critic warn others to hold off putting this project to the test until 'adults' have reviewed it.
- And then finally you suggest they do it anyway, with your permission this time (and of course now amply warned) with the implicit assumption that problems will turn up (most likely this will be the case) and that you hope 'there won't be too many false positives', strongly suggesting that there might be.
And in your comment prior to this reply you do that once again, making statements that put words in the mouth of the author.
It's difficult for me to have a positive opinion of the author when he responds with dismissal and derision to concerns others have raised about Fil-C and memory safety under data races.
The fact is that Fil-C allows capability and pointer writes to tear. That is, when thread 1 writes pointer P2 to a memory location previously holding P1, thread 2 can observe, briefly, the pointer P2 combined with the capability for P1 (or vice versa, the capability for P2 coupled to the pointer bits for P1).
Because thread 2 can observe a mismatch between a pointer and its capability, an attacker controlled index into P2 from thread 2 can access memory of an object other than the one to which P2 points.
The mismatch of pointer and capability breaks memory safety: an attacker can break the abstraction of pointers-as-handles and do nefarious things with pointers viewed instead as locations in RAM.
On one hand, this break is minor and doesn't appear when memory access is correctly synchronized. Fil-C is plenty useful even if this corner case is unsafe.
On the other hand, the Fil-C as author's reaction to discourse about this corner case makes me hesitant to use his system at all. He claims Java has the same problem. It does not. He claims it's not a memory safety violation because thread 1 could previously have seen P1 and its capability and therefore accessed any memory P1's capability allowed. That's correct but irrelevant: thread 2 has P2 and it's paired with the wrong capability. Kaboom.
The guy is technically talented, but he presents himself as Prometheus bringing the fire of memory safety to C-kind. He doesn't acknowledge corner cases like the one I've described. Nor does he acknowledge practical realities like the inevitability of some kind of unsafe escape hatch (e.g. for writing a debugger). He says such things are unnecessary because he's wrapped every system call and added code to enforce his memory model's invariants around it. Okay, is it possible to do that in the context of process_vm_writev?
I hope, sincerely, the author is able to shift perspectives and acknowledge the limitations of his genuinely useful technology. The more he presents it as a panacea, the less I want to use it.
> Because thread 2 can observe a mismatch between a pointer and its capability, an attacker controlled index into P2 from thread 2 can access memory of an object other than the one to which P2 points.
Under Fil-C’s memory safety rules, „the object at which P points” is determined entirely by the capability and nothing else.
You got the capability for P1? You can access P1. That’s all there is to it. And the stores and loads of the capability itself never tear. They are atomic and monotonic (LLVM’s way of saying they follow something like the JMM).
This isn’t a violation of memory safety as most folks working in this space understand it. Memory safety is about preventing the weird execution that happens when an attacker can access all memory, not just the memory they happen to get a capability to.
> He claims Java has the same problem. It does not.
It does: in Java, what object you can access is entirely determined by what objects you got to load from memory, just like in Fil-C.
You’re trying to define „object” in terms of the untrusted intval, which for Fil-C’s execution model is just a glorified index.
Just because the nature of the guarantees doesn’t match your specific expectations does not mean that those guarantees are flawed. All type systems allow incorrect programs to do wrong things. Memory safety isn’t about 100% correctness - it’s about bounding the fallout of incorrect execution to a bounded set of memory.
> That's correct but irrelevant: thread 2 has P2 and it's paired with the wrong capability. Kaboom.
Yes, kaboom. The kaboom you get is a safety panic because a nonadversarial program would have had in bounds pointers and the tear that arises from the race causes an OOB pointer that panics on access. No memory safe language prevents adversarial programs from doing bad things (that’s what sandboxes are for, as TFA elucidates).
But that doesn’t matter. What matters is that someone attacking Fil-C cannot use a UAF or OOBA to access all memory. They can only use it to access whatever objects they happen to have visibility into based on local variables and whatever can be transitively loaded from them by the code being attacked.
That’s memory safety.
> He doesn't acknowledge corner cases like the one I've described.
You know about this case because it’s clearly documented in the Fil-C documentation. You’re just disagreeing with the notion that the pointer’s intval is untrusted and irrelevant to the threat model.
You don't always get a panic. An attacker who can get a program to access an offset he controls relative to P2 can access P1 if P2 is torn such that it's still coupled, at the moment of adversarial access, with P1's capability. That's dangerous if a program has made a control decision based on the pointer bits being P2. IOW, an attacker controlled offset can transform P2 back into P1 and access memory using P1's capability even if program control flow has proceeded as though only P2 were accessible at the moment of adversarial access.
That can definitely enable a "weird execution" in the sense that it can let an attacker make the program follow an execution path that a plain reading of the source code suggests it can't.
Is it a corner case that'll seldom come up in practice? No. Is it a weakening of memory safety relative to what the JVM and Rust provide? Yes.
You are trying to define the problem away with sleigh-of-hand about the pointer "really" being its capability while ignoring that programs make decisions based on pointer identity independent of capability -- because they're C programs and can't even observe these capabilities. The JVM doesn't have this problem, because in the JVM, the pointer is the capability.
It's exactly this refusal to acknowledge limitations that spooks me about your whole system.
> An attacker who can get a program to access an offset he controls relative to P2 can access P1 if P2 is torn such that it's still coupled, at the moment of adversarial access, with P1's capability
Only if the program was written in a way that allowed for legitimate access to P1. You’re articulating this as if P1 was out of thin air; it’s not. It’s the capability you loaded because the program was written in a way that let you have access to it. Like if you wrote a Java program in a way where a shared field F sometimes pointed to object P1. Of course that means loaders of F get to access P1.
> That can definitely enable a "weird execution"
Accessing a non-free object pointed by a pointer you loaded from the heap is not weird.
I get the feeling that you’re not following me on what „weird execution” is. It’s when the attacker can use a bug in one part of the software to control the entire program’s behavior. Your example ain’t that.
> Is it a corner case that'll seldom come up in practice? No. Is it a weakening of memory safety relative to what the JVM and Rust provide? Yes.
I don’t care about whether it’s a corner case.
My point is that there’s no capability model violation and no weird execution in your example.
It’s exactly like what the JVM provides if you think of the intval as just a field selector.
I’m not claiming it’s like what rust provides. Rust has stricter rules that are enforced less strictly (you can and do use the unsafe escape hatch in rust code to an extent that has no equal in Fil-C).
I think his argument is that you can have code this:
user = s->user;
if(user == bob)
user->acls[s->idx]->has_all_privileges = true;
And this happens:
1. s->user is initialized to alice
2. Thread 1 sets s->idx to ((alice - bob) / sizeof(...)) and s->user to Bob, but only the intval portion is executed and the capability still points to Alice
3. Thread 2 executes the if, which succeeds, and then gives all privileges to Alice unexpectedly since the bob intval plus the idx points to Alice, while the capability is still for Alice
It does seem a real issue although perhaps not very likely to be present and exploitable.
Seems perhaps fixable by making pointer equality require that capabilities are also equal.
1. I’m not claiming that Fil-C fixes all security bugs. I’m only claiming that it’s memory safe and I am defining what that means with high precision. As with all definitions of memory safety, it doesn’t catch all things that all people consider to be bad.
2. Your program would crash with a safety panic in the absence of a race. Security bugs are when the program runs fine normally, but is exploitable under adversarial use. Your program crashes normally, and is exploitable under adversarial use.
So not only is it not likely to be present or exploitable, but if you wrote that code then you’d be crashing in Fil-C in whatever tests you ran at your desk or whenever a normal user tried to use your code.
But perhaps point 1 is still the most important: of course you can write code with security bugs in Fil-C, Rust, or Java. Memory safety is just about making a local bug not result in control of arbitrary memory in the whole program. Fil-C achieves that key property here, hence its memory safe.
> I’m only claiming that it’s memory safe and I am defining what that means with high precision
Do you have your definition of memory safety anywhere? Specifically one that's precise enough that if I observe a bug in a C program compiled via Fil-C, I can tell whether this is a Fil-C bug allowing (in your definition) memory unsafety (e.g. I'm pretty sure an out-of-bounds read would be memory unsafety), or if it's considered a non-memory-safety bug that Fil-C isn't trying to prevent (e.g. I'm pretty sure a program that doesn't check for symlinks before overwriting a path is something you're not trying to protect against). I tried skimming your website for such a definition and couldn't find this definition, sorry if I missed it.
I typically see memory safety discussed in the context of Rust, which considers any torn read to be memory-unsafe UB (even for types that don't involve pointers like `[u64; 2]`, such a data race is considered memory-unsafe UB!), but it sounds like you don't agree with that definition.
In my understanding the program can work correctly in normal use.
It is buggy because it fails to check that s->idx is in bounds, but that isn't problem if non-adversarial use of s->idx is in bounds (for example, if the program is a server with an accompanying client and s->idx is always in bounds when coming from the unmodified client).
It is also potentially buggy because it doesn't use atomic pointers despite comcurrent use, but I think non-atomic pointers work reliably on most compiler/arch combinations, so this is commonplace in C code.
A somewhat related issue if that since Fil-C capabilities currently are only at the object level, such an out-of-bounds access can access other parts of the object (e.g. an out-of-bounds access in an array contained in an array element can overwrite other either of the outer array)
It is true though that this doesn't give arbitrary access to any memory, just to the whole object referred to by any capability write that the read may map to, with pointer value checks being unrelated to the accessed object.
If you set the index to `((alice - bob) / sizeof(...))` then that will fail under Fil-C’s rules (unless you get lucky with the torn capability and the capability refers to Alice).
Exactly. I agree that this specific problem is hard to exploit.
> Seems perhaps fixable by making pointer equality require that capabilities are also equal
You'd need 128-bit atomics or something. You'd ruin performance. I think Fil-C is actually making the right engineering tradeoff here.
My point is that the way Pizlo communicates about this issue and others makes me disinclined to trust his system.
- His incorrect claims about the JVM worry me.
- His schtick about how Fil-C is safer than Rust because the latter has the "unsafe" keyword and the former does not is more definitional shenanigans. Both Fil-C and Rust have unsafe code: it's just that in the Fil-C case, only Pizlo gets to write unsafe code and he calls it a runtime.
What other caveats are hiding behind Pizlo's broadly confident but narrowly true assertions?
I really want to like Fil-C. It's good technology and something like it can really improve the baseline level of information security in society. But Pizlo is either going to have to learn to be less grandiose and knock it off with the word games. If he doesn't, he'll be remembered not as the guy who finally fixed C security but merely as an inspiration for the guy who does.
All I’m really hearing is that this guy rubs you the wrong way, so you’re not going to give him the benefit of the doubt that you’d give to others.
I mean, maybe you’re right that his personality will turn everyone off and none of this stuff will ever make it upstream. But that kind of seems like a problem you’re actively trying to create via your discourse.
> Only if the program was written in a way that allowed for legitimate access to P1. You’re articulating this as if P1 was out of thin air; it’s not.
My program:
if (p == P2) return p[attacker_controlled_index];
If the return statement can access P1, disjoint from P2, that's a weird execution for any useful definition of "weird". You can't just define the problem away.
Your central claim is that you can take any old C program, compile it with Fil-C, and get a memory-safe C program. Turns out you get memory safety only if you write that C program with Fil-C's memory model and its limits in mind. If someone's going to do that, why not write instead with Rust's memory model in mind and not pay a 4x performance penalty?
> that's a weird execution for any useful definition of "weird".
Weird execution is a term of art in the security biz. This is not that.
Weird execution happens when the attacker can control all of memory, not just objects the victim program rightly loaded from the heap.
> Your central claim is that you can take any old C program, compile it with Fil-C, and get a memory-safe C program.
Yes. Your program is memory safe. You get to access P1 if p pointed at P1.
You don’t get to define what memory safety means in Fil-C. I have defined it here: https://fil-c.org/gimso
Not every memory safe language defines it the same way. Python and JavaScript have a weaker definition since they both have powerful reflection including eval and similar superpowers. Rust has a weaker definition if you consider that you can use `unsafe`. Go has a weaker definition if you consider that tearing in Go leads to actual weird execution (attacker gets to pop the entire Go type system). Java’s definition is most similar to Fil-C’s, but even there you could argue both ways (Java has more unsafe code in its implementation while Fil-C doesn’t have the strict aliasing of Java’s type system).
You can always argue that someone else’s language isn’t memory safe if you allow yourself to define memory safety in a different way. That’s not a super useful line of argumentation, though it is amusing and fun
You may define "memory safety" as you like. I will define "trustworthy system" as one in which the author acknowledges and owns limitations instead of iteratively refining private definitions until the limitations disappear. You can define a mathematical notation in which 2+3=9, but I'm under no obligation to accept it, and I'll take the attempt into consideration when evaluating the credibility of proofs in this strange notation.
Nobody is trying to hide the existence of "eval" or "unsafe". You're making a categorical claim of safety that's true only under a tendentious reading of common English words. Users reading your claims will come away with a mistaken faith in your system's guarantees.
> I will define "trustworthy system" as one in which the author acknowledges and owns limitations instead of iteratively refining private definitions until the limitations disappear.
You know about this limitation that you keep going on about because it’s extremely well documented on fil-c.org
[Woman walking on beach at sunset, holding hands with husband]
Voiceover: "Miracurol cures cancer."
[Couple now laughing over dinner with friends]
"Ask your doctor if Miracurol is right for you."
[Same footage continues, voice accelerates]
"In clinical trials, five mice with lymphoma received Miracurol. All five were cured. One exploded. Not tested in humans. Side effects include headache, itchiness, impotence, explosion, and death. Miracurol's cancer-free guarantee applies only to cancers covered under Miracurol's definition of cancer, available at miracurol.org. Manufacturer not responsible for outcomes following improper use. Consult your doctor."
[Couple walking golden retriever, sun flare]
Voiceover: "Miracurol. Because you deserve to live cancer-free."
Patient: "I exploded."
Miracurol: "That's extremely well documented on miracurol.org."
It’s really sad to see your posts on this thread. Fil-C is an incredible achievement and absolutely full of interesting technical details to dig into. I’m not a mod, but as a reader of the site and someone who takes a curious interest in the progress of Fil-C, can you please stop attacking its creator like this. It’s tedious, needlessly rude, and lessens the opportunity for the rest of us to actually learn something from an expert.
Facts are facts and exist independent of who discovers them. If you'd like to learn, the last thing you want to do is stop people poking at contradictions and pressure-testing claims. If Fil-C is really the "incredible achievement" you say it is, it can withstand scrutiny.
> I will define "trustworthy system" as one in which the author acknowledges and owns limitations
You can't then go on to complain that the author does document the limitations but considers the overall system good. Fil-C, by the definition you just espoused, is a "trustworthy system".
Sorry to intrude on the discussion, but I have a hard time grasping how to produce the behavior mentioned by quotemstr. From what I understand the following program would do it:
int arr1[] = {1, 2, 3, 4, 5};
int arr2[] = {10, 20, 30, 40, 50};
int *p1 = &arr1[1];
int *p2 = &arr2[2];
int *p = choose_between(p1,p2);
//then sometime later, a function gets passed p
// and this snippet runs
if (p == p2) {
//p gets torn by another thread
return p; // this allows an illegal index/pointer combo, possibly returning p1[1]
}
Is this program demonstrating the issue? Does this execute under Fil-C's rules without a memory fault? If not, could you provide some pseudocode that causes the described behavior?
Fil-C lets programs access objects through the wrong pointer under data race. All over the Internet, you've responded to the tearing critique (and I'm not the only one making it) by alternatively 1) asserting that racing code will panic safely on tear, which is factually incorrect, and 2) asserting that a program can access memory only through its loaded capabilities, which is factually correct but a non sequitur for the subject at hand.
You're shredding your credibility for nothing. You can instead just acknowledge Fil-C provides memory safety only for code correctly synchronized under the C memory model. That's still plenty useful and nobody will think less of you for it. They'll think more, honestly.
> asserting that racing code will panic safely on tear, which is factually incorrect
Try it. That’s what happens.
> through its loaded capabilities, which is factually correct but a non sequitur for the subject at hand.
It’s literally the safety property that Fil-C guarantees.
Safety properties provided by languages aren’t about preventing every bad thing that users can imagine. Just because the language does something different than what you expect - even if it allows you to write a program with a security bug - doesn’t mean that the language in question isn’t memory safe.
> You're shredding your credibility for nothing. You can instead just acknowledge Fil-C provides memory safety only for code correctly synchronized under the C memory model.
Fil-C provides memory safety even for incorrectly synchronized code. That safety guarantee is easy to understand and easy to verify: you only get to access the memory of the capability you actually loaded. You’re trying to evade this definition by getting hung up on what the pointer’s intval was, and your PoC uses a pointer comparison to illustrate that. You’re right that the intval is untrusted under Fil-C rules.
I’m not going to downplay the guarantees of my technology just to appease you. Whether or not you find me credible is less important to me than being honest about what Fil-C guarantees.
> If you set the index to `((alice - bob) / sizeof(...))` then that will fail under Fil-C’s rules (unless you get lucky with the torn capability and the capability refers to Alice).
In the comment above, you write, referring to a fault on access through a torn capability
> Try it. That’s what happens.
Your position would be clearer if you could resolve this contradiction. Yes or no: does an access through a pointer with an arbitrary offset under a data race that results in that pointer's capability tearing always fault?
> You’re right that the intval is untrusted under Fil-C rules.
Can Fil-C compile C?
You can't argue, simultaneously,
1) it's the capability, not your "intval", that is the real pointer with respect to execution flow and simultaneously, and
2) that Fil-C compiles normal C in which the "intval" has semantic meaning.
Your argument is that Fil-C is correct with respect to capabilities even if pointers are transiently incorrect under data races. The trouble is that Fil-C programs can't observe these capabilities and can observe pointers, and so make control flow decisions based on these transient incorrect (you call them "untrusted") inputs.
Can you show an actual minimal C program which has this problem? I’m trying to follow along here, but it’s very hard for me to understand the exact scenario you’re talking about.
I think at this point it's reasonable to conclude that quotemstr does not have a legitimate concern until a program demonstrating the issue can be presented.
> Rust has a weaker definition if you consider that you can use `unsafe`
I don't see it. Rust makes the same guarantees regardless of the unsafe keyword. The difference is only that with the unsafe keyword you the programmer are responsible for upholding those guarantees whereas the compiler can check safe Rust.
But the definition is what we're talking about, not whether you make mistakes. Of course it's important that safe Rust is checked by the compiler, but that's crucially not part of how safety is defined.
I would guess that somebody more on the pulse of C's safety efforts could tell you whether they have a definition of memory safety for C or whether they're comfortable with an existing definition from somebody else.
What I mean is, what’s to stop us saying that C upholds all the same guarantees that Rust does and that it’s the programmer that’s responsible for upholding them (just as the programmer is responsible in the case of Rust code marked ‘unsafe’)? This seems like a semantic game to avoid acknowledging that unsafe Rust comes with some of (though not all) of the same risks as C code.
In short, the definitions are not important. What matters are the risks that you do or don’t run. And if your Rust code contains unsafe blocks, you are running risks that you wouldn’t be if you used Fil-C, which has no such escape hatch. (Of course this goes both ways – your Fil-C code is more likely to fail, safely, with a runtime error due to a mistake that Rust would have caught at compile time.)
Real world C software does not read like software written by people who are in fact upholding those guarantees you say C could equally have. It reads as though they think such a guarantee is a joke or an irrelevance. It's not rare for me to run into people who think C's pointers are just indexing into a massive array of all RAM (or its equivalent on today's systems with virtual addressing), that's not just not in the same ballpark as a safe C program, that's playing a different sport on another continent.
You seem to be suggesting that a language being safe or unsafe is a social contract rather than a technical property of the language.
>And do you say that C offers these guarantees ?
No, that would be silly, and it's an illustration of why it is silly to say that a language guarantees X if it is the programmer who must check that X holds. If we go down that route (which, to repeat, would be silly), then we can make C safe without any technical changes just by adding some language to the standard saying that C programmers are obliged to ensure that their code maintains a certain list of invariants. When you say that "Rust makes the same guarantees regardless of the unsafe keyword", it seems to me that you are doing something equally pointless.
> You seem to be suggesting that a language being safe or unsafe is a social contract rather than a technical property of the language.
Quite some way up this thread pizlonator insists that each programming language defines memory safety differently, quantifying some as "weaker" or "stronger" and giving the example that Rust has the `unsafe` keyword and so that's weaker than Fil-C.
That's what we were discussing when you jumped in with your C hypothetical.
You apparently instead believe in a single universal "safety" and every language is either absolutely safe or unsafe according to foldr for whatever that's worth - but that's not what we were talking about.
No, I just think that Rust is less safe than it would be if it didn’t have the unsafe escape hatch.
I think you’re taking issue with how pizlonator phrased his post rather than addressing the substance of his point that Fil-C does not have the ‘unsafe’ escape hatch and is therefore safer in this respect. Sure, Rust uses a pretty standard definition of memory safety when talking about the desired property of the program, but pizlonator is talking about the definition of memory safety that the Rust compiler actually guarantees that Rust code will meet, which (when you include unsafe-marked code) is a conditional and weaker one.
I still can't agree with weaker. Yes, it's conditional on the unsafe code actually obeying the rules, and on the tooling, but Fil-C has the same situation, there will be bugs in the compiler, indeed in some cases the same bugs because LLVM has plenty of bugs.
Crucially unsafe Rust doesn't have weaker rules, it has the same rules, that's my whole thrust here. The change is that the burden of obeying those rules is on you, the programmer, and that's a heavy burden. But it is very much possible for skilled practitioners to carefully achieve that. It's very similar skill to writing C++ stdlib implementations. Aria's "Pre-pooping your pants" essay is colourful but ultimately it's the same sort of thing strong exception guarantees are made of in C++. We go in eyes open, expecting the worst so that we're pleasantly surprised when it doesn't happen.
It's not practical for humans to write code like this all day, everyday, they make too many mistakes that's the problem in C or C++ - but, seems like it is practical for some skilled people, sometimes, with the benefit of oversight from similarly skilled peers.
If that isn't enough for you I have good news and I have bad news. The bad news is that for general purpose software too bad, we've known since the middle of last century that we can't do better than this. Fil-C isn't magic, nor are Java and C#. At runtime we detect we can no longer assure correct operation and we abort, this might not be OK, but we can't do better.
The good news is that we can do better if we're willing to sacrifice generality. The difficulties all come from the fact that ultimately if we're a general purpose programming language we can be made into a Gödel number and then obliged to perform computations on ourselves and we're toast. But we can invent useful languages which aren't powerful enough to do that. Want to decompress ZIP files? No generality needed. Crop some JPEGs? Not needed. Validate whether these graphs are isomorphic? Ditto.
Rust is a general purpose language, but you might well not need one for your problem, and I say we should prefer not to use a general purpose language when we don't need one.
I don't think it makes sense to compare non-invariant-respecting unsafe blocks to compiler bugs. It would make sense to do so if unsafe blocks were only present in a highly-verified Rust stdlib, but we both know that's not the case.
>Crucially unsafe Rust doesn't have weaker rules, it has the same rules, that's my whole thrust here. The change is that the burden of obeying those rules is on you, the programmer, and that's a heavy burden.
Now we're circling back to my argument about C. The C standards committee could declare that 'unsafe C' (i.e. all C) has these very same rules (which C programmers have the heavy burden of obeying). Would this instantly convert C into a memory safe language? Of course not! It's an empty semantic gesture. Similarly, merely saying "Rust programmers are obliged to respect the following invariants inside unsafe blocks!" does nothing to actually decrease the risks associated with unsafe blocks (leaving aside whatever exhortive success such admonitions might have).
What next, if we accept this logic? Is Perl a language with strict static typing, but "the burden of checking the types falls on you, the programmer"?.
> Would this instantly convert C into a memory safe language?
I actually strongly encourage finishing this thought. Imagine you're WG14 and you're intending to drop this "safe C" bombshell in C29 and think about the implications just for your document.
You "just" need to spell out all these rules for the language itself, Rust only has a handful but you've got pages of these, and then you need to go re-design all your APIs so that instead of basically YOLO† each of the standard library APIs has an explicable set of safety requirements, just as Rust has to do with its relatively small subset of unsafe APIs.
A subset of this work is already underway for WG14 and has been for a few years. You'll need to hurry them along to prepare for your epoch making announcement because they're not expecting to be anywhere close by 2029 but hey, shoot for the stars right?
At the end of this, you've announced a deeply incompatible C version and the benefit is that if your customers can hire people who don't make mistakes and they port all their C to this new version, it has similar properties to if they were to rewrite it in Rust. Don't expect applause, in fact, I'd recommend hiring bodyguards.
† I think people really underestimate how much C relies on this. Remember C provides qsort, an unstable type-erased comparison sort (hopefully an introsort, but in some implementations literally just Hoare's Quicksort which is significantly older than C itself) which has arbitrary Undefined Behaviour if you screwed up your comparison function and yet for all the popular implementations it's still slower than in Rust which doesn't have that UB problem at all.
So now you're documenting specifically for functions like this why "safe C" is both much harder to use and slower in your standards document like it's an achievement, and unlike Karoline Leavitt you're not even getting paid to do this. Maybe you should take up knitting instead?
You’ve kindly finished the thought for me. You now have a “safe” version of C purely by updating the standard and associated documentation. (It’s not actually true that you need to redesign all the APIs in the stdlib. You just need to document the appropriate restrictions on how they may safely be used, just as you would have to do with a Rust function marked ‘unsafe’. It’s trivially possible to write an unsafe-marked Rust function that’s guaranteed to be memory safe if and only if an arbitrary invariant is maintained.)
In reality of course, this is all absurd. No matter how much or how little work it turns out to be, writing reams of standardese and leaving actual C implementations untouched would do nothing to reduce the safety risks associated with C code.
By the way, you’re wasting some energy arguing points that I agree with (e.g. that Rust has a better sorting API than C). My comments here are not anti-Rust. I merely disagree with the claim that Rust code marked ‘unsafe’ is as safe as regular Rust code.
> ... then we can make C safe without any technical changes just by adding some language to the standard saying that C programmers are obliged to ensure that their code maintains a certain list of invariants.
In Rust you can use #![forbid(unsafe_code)]
to totally forbid unsafe code in your codebase. Rust also checks for memory safety at compile time, these are strong guarantees that ensure that if the code compiles it is memory safe.
I'm aware of that, but I'm responding to the original claim that "Rust makes the same guarantees regardless of the unsafe keyword" (see https://news.ycombinator.com/item?id=46262774)
Ah. I agree with you. When unsafe is used the borrow checker cannot check for memory safety, the programmer has to provide the guarantees by making sure their code does not violate memory safety, similar to programming in C.
But unsafe Rust is still far better than C because the unsafe keyword is visible and one can grep it and audit the unsafe parts. Idiomatic Rust also requires that the programmer provides comments as to why that part is unsafe.
I think making things more explicit with "unsafe" is an advantage of Rust, but I think "far better" is a bit of an exaggeration. In C you need to audit pointer arithmetic, malloc/free, casts and unons. If you limit pointer arithmetic to a few safe accessor functions and have a documented lifetime rules, this is also relatively simple to do (more difficult than "grep" but not much). Vice versa, if you use a lot of "unsafe" in Rust or in complicated ways, it can also easily become possible to guarantee safety. In contrast to what people seem to believe, the bug does not need to be inside in unsafe block (a logic error outside can cause the UB inside unsafe or a violation of some of Rust's invariants inside unsafe can allow UB outside of unsafe) and can result even from the interaction of unsafe blocks.
The practical memory safety we see in Rust is much more the result of trying hard to avoid memory safety issues and requiring comments for unsafe blocks is part of this culture.
> the bug does not need to be inside in unsafe block
The argument is that while you wouldn't in fact fix the bug by modifying the unsafe code block, the unsafe code block was wrong until you fixed the other code.
For example imagine if a hypothetical typo existed inside RawVec (the implementation details of Vec) causing the growable array to initially believe it has 1 element inside it, not 0 even though no space has been allocated and nothing was stored. That's safe code, and of course the correct fix would be to change it from 1 to 0, easy. But this broken type is arguably broken because the unsafe code would deference a pointer that isn't valid, trying to reach that non-existent value. It would be insane, perhaps even impossible, to modify that code to somehow handle the "We wrote 1 instead of 0" mistake, when you could instead fix the bug - but that is where the theoretical fault lies.
I'm curious what you make of quotemastr's point about a race causing a mismatch between the pointer's capability and its index. First off, in your estimation can this realistically be exploited to wreak havoc on extant C programs compiled using Fil-C? Second, is such a mismatch able to happen in safe Rust? Third, is such a mismatch able to happen in unsafe Rust?
Edit: clarification to narrow the question even further
"Wreak havoc" is a very vague claim. Instinctively the tearing feels like something very difficult to usefully exploit, but, we know historically that the only people who can reliably tell you whether it was difficult are the attackers actually trying to do it. Don't believe the defenders.
AIUI this capability versus value distinction is a Fil-C thing. So, that's not a thing in Rust at all. In Safe Rust the pointer types, which is what we care about here, aren't very interesting because safe Rust can't dereference them, safe Rust is fine with you making a pointer from the word "LAUGHING" (not a pointer to the string, just the literal bytes in ASCII, but treated as a pointer) or from just some random bytes you found in a data file, because it's not allowed to dereference them so, cool, whatever, no harm no foul.
In unsafe Rust we're allowed to dereference valid pointers, but it's our job to ensure we obey that rule about validity, it being our job to obey rules is what "unsafe" means. So, that silly "LAUGHING" pointer isn't valid, it's just pointer-shaped toxic material. Even if, by coincidence, a pointer you have happened to have the same address as that pointer, in both C and Rust it's not OK to just go around dereferencing invalid pointers, they are not offsets into an imaginary huge array of all memory even though some C programmers act like they are.
Ignoring the Fil-C specific capabilities, in Rust the tearing issue is a matter of synchronization, which is something Rust cares about as part of delivering "fearless concurrency". Rust's marker traits Send and Sync are good place to start learning about that. Yes, we could unsafely implement these marker traits in unsafe Rust when we shouldn't, and thus enable what I imagine you'd call havoc.
So, mostly the problem is that your question is (unintentionally) too vague to answer well but I hope I was at least somewhat helpful.
I'm not an expert here but I have to say this feels like a very weak objection.
p points to P1. One thread reads through p. Another thread races with that and mutates p to point to P2. The result is the first thread reads from either P1 or P2 (but no other object).
This seems totally fine and expected to me? If there's a data race on a pointer, you might read one or the other values, but not garbage and not out of bounds. I mean, if it could guarantee a panic that's nice, but that's a bonus, not required for safety.
Posts like the one I made about how to do sandboxing are specifically to make the runtime transparent to folks so that meaningful auditing can happen.
> For example, Filip mentioned that some setuid programs can be compiled with it, but it also makes changes to ld.so. I pointed this out to the author on Twitter, as it could be problematic.
The changes to ld.so are tiny and don’t affect anything interesting to setuid. Basically it’s just one change: teaching the ld.so that the layout of libc is different.
More than a month ago, I fixed a setuid bug where the Fil-C runtime was calling getenv rather than secure_getenv. Now I’m just using secure_getenv.
> In other words, these are still teething problems with Fil-C, which will be reviewed and fixed over time. I just want to point out that using it for real-world "infrastructures" might be somewhat risky at this point. We need unix nerds to experiment with.
There’s some truth to what you’re saying and there’s also some FUD to what you’re saying. Like a perfectly ambiguous mix of truth and FUD. Good job I guess?
Is it FUD? Approximately speaking, all software has bugs. Being an early adopter for security critical things is bound to carry significant risk. It seems like a relevant topic to bring up in this sort of venue for a project of this sort.
It's true. I used to promote high-assurance kernels. They had low odds of coding errors but the specs could be wrong. Many problems Linux et al. solved are essentially spec-level. So, we just apply all of that to the secure designs, right?
Well, those spec issues are usually not documented or new engineers won't know where to find a full list. That means the architecturally-insecure OS's might be more secure in specific areas due to all the investment put into them over time. So, recommending the "higher-security design" might actually lower security.
For techniques like Fil-C, the issues include abstraction gap attacks and implementation problems. For the former, the model of Fil-C might mismatch the legacy code in some ways. (Ex: Ada/C FFI with trampolines.) Also, the interactions between legacy and Fil-C might introduce new bugs because integrations are essentially a new program. This problem did occur in practice in a few, research works.
I haven't reviewed Fil-C. I've forgotten too much C and the author was really clever. It might be hard to prove the absence of bugs in it. However, it might still be very helpful in securing C programs.
My trouble with separate categories "memory safety technology" and "sandboxing technology" is that something like WASM execution is both:
* Depending on how WASM is used, one gets safety guarantees. For example, memory is not executable.
* Privileges are reduced as a WASM module interacts with the environment through the WASM runtime and the embedder
Now, when one compiles C to WASM one may well compile things with bugs. A memory access bug in C is still a memory access bug, but its consequences can be limited in WASM execution. Whether fail-stop behavior is guaranteed actually depends on the code the C compiler generates and the runtime (allocation/deallocation, concurrency) it sets up.
So when we enumerate immediately available security options and count WASM as sandboxing, this is not wrong. But WASM being an execution environment, one could do a lot of things, including a way of compiling and executing C that panics when a memory access bug is encountered.
Say your C program has sensitive information in module A and a memory safety bug in module B. Running that program in wasm won’t prevent the attacker from using the bug in B to get read/write access to the data in A.
In practice what the attacker will really do is use the memory safety bug to achieve weird execution: even without control over the program counter, the fact that a memory safety bug inside the wasm memory gives read write access to all of that memory means the attacker can make the program do whatever they want, subject to the wasm sandbox limits (ie whatever the host allows the wasm guest to do).
Basically wasm amounts to a lightweight and portable replacement for running native code in a sufficiently sandboxed process
Your general point stands - wasm's original goal was mainly sandboxing - but
1. Wasm does provide some amount of memory safety even to compiled C code. For example, the call stack is entirely protected. Also, indirect calls are type-checked, etc.
2. Wasm can provide memory safety if you compile to WasmGC. But, you can't really compile C to that, of course...
Correct me if I'm wrong, but with LLVM on Wasm, I think casting a function pointer to the wrong type will result in you calling some totally unrelated function of the correct type? That sounds like the opposite of safety to me.
I agree about the call stack, and don't know about GC.
That is incorrect about function pointers: The VM does check that you are calling the right function type, and it will trap if the type does not match.
Here it is in the spec:
> The call_indirect instruction calls a function indirectly through an operand indexing into a table that is denoted by a table index and must have type funcref. Since it may contain functions of heterogeneous type, the callee is dynamically checked against the function type indexed by the instruction’s second immediate, and the call is aborted with a trap if it does not match.
(Other sandboxing approaches, including related ones like asm.js, do other things, some closer to what you mentioned. But wasm has strict checking here.)
Depends on how it is used is already a sign that WebAssembly isn't really as safe as being sold, by many of its advocates, versus other bytecode formats.
Like, C is actually really safe, it only depends on how it is being used.
People only have to enumerate the various ways and tools to write safe code in C.
WASM is just a bytecode format for a stack based vm. Granted it is weirdly named, the actual "Assembly" equivalent is WAT.
But the point is, it is a format specification, which has nothing to do with safety. You can implement a totally unsafe WASM runtime if you so choose. Personally I think it's not a bad thing, at least we have something like it that can run in a browser environment. But I am curious to know why you dislike it so much.
> including a way of compiling and executing C that panics when a memory access bug is encountered.
WASM couldn’t do that because it doesn’t have a sense of the C memory model nor know what is and isn’t safe - that information has long been lost. That kind of protection is precisely what Fil-C is doing.
WASM is memory safe in that you can’t escape the runtime. It’s not memory safe in that you can escape escape the program running within the sandbox, which you can’t do with a memory safe language like Rust or Fil-C.
It would require a bit of porting (since Fil-C currently assumes you have all of the Linux syscalls). But you could probably even lift some of the microVM’s functionality into Fil-C’s memory safe userland.
I hope this project gets more traction. I would love to see a memory safe battle tested sudo or polkit in my package manager without having to install a potentially workflow breaking replacement.
If you're into Nix, check out https://github.com/mbrock/filnix — not yet integrated & maintained in upstream Nixpkgs, but lets you replace Nix/NixOS packages with Fil-C versions quite easily.
Which requirements does a full blown virtual machine not meet? By leaning on that as the sandbox, we get Qubes, but maybe I don't know what I'm talking about.
It’s true that a full blown VM is an excellent sandbox.
The usual situation is like what chrome or OpenSSH want:
- They want to be able to do dangerous things by design. Chrome wants to save downloads. Chrome wants to call rendering APIs. OpenSSH wants to pop a shell.
- They want to deal with untrusted inputs. Chrome downloads things off the internet and parses them. OpenSSH has a protocol that it parses.
So you want to split your process into two with privilege separation:
- one process has zero privileges and does the parsing of untrusted inputs.
- another process has high privilege but never deals with untrusted inputs.
And then the two processes have some carefully engineered IPC protocol for talking to one another.
Could you run the deprivileged process in a VM for maximum security? Yeah that’s one way to do it. But it’s cleaner to run it as a normal process, ask the OS to sandbox it (deprivilege it), and then have a local domain socket (or whatever) that the two processes can use to communicate.
If you used a VM for deprivileging then:
- There’d be more overhead. Chrome wants to do this per origin per tab. OpenSSH wants to do it per connection. Maybe a VM is too much
- You could put the whole browser into the VM but then you’d still need something outside it for saving files. And probably for talking to the GPU. You could run OpenSSH in the VM but then that defeats the purpose (you want to use it to pop a shell after all).
- You can use vsocks and other things to communicate between host and guest but it’s much more gross than the options available when using traditional process sandboxing
Does it even work with openssh example? Pwning the parser progress will let attacker spoof arbitrary communication, which in case of SSH lets them execute arbitrary commands. Or is there a smart way to work around that?
OS-level sandboxes are way too coarse grained to achieve a good "hollowing out" of the attack surface. The principle of least privilege should extend down to/start at the individual language library level (because this is where the actual trust boundaries are), or even finer grained, at the individual function or code segment level (thereby providing maximum control), and therefore not be limited to larger domains.
Most software today relies on many (imported, third party) libraries, so the security architecture should provide primitives/abstractions to manage rights at that level, which requires programming languages to implement the ability to sandbox (managing the effects of) code. If they did this with lightweight, portable virtual machines like WebAssembly, that could work.
The vast majority of code out there should be limited to pure computation and have no ability to access anything external at all (and otherwise, only what it actually requires) - yet most languages are simply incapable of providing any such guarantees. If the programmer of software cannot get ironclad assurances, they cannot in turn provide them to their users.
I'm not saying that OS-level sandboxing isn't good, just that it doesn't go far enough. And depending on the setup, it may not sufficiently limit the effects of compromised elements, and it provides no "monitoring in the small". It's also not convenient or efficient to have an entire OS instance for every single system component. Compartmented microkernel operating systems like Genode do it better imo.
When it comes to VMs, most things are solved and have near native performance, but desktop graphics are not. Due to limitations in GPU architecture, you usually have to dedicate an entire GPU to the VM to have reasonable graphical acceleration. Qubes doesn't solve this either, IIRC the apps running in VMs are glued to the host with X11 forwarding without any acceleration support.
For what it's worth the Intel B50 supports GPU virtualization and software support seems to be improving quickly. Really hoping this blows the lid off over the next couple years.
Nit：The word “orthogonal” should not mean merely “different”. It should mean “completely unrelated” if we are drawing a proper analogy from linear algebra. Orthogonal vectors have a dot product of zero. No correlation whatsoever. As ML and linear algebra terms spread to more common language of course the terms will change their meaning. Just as “literally” now often means “figuratively” I’m not going to die on this hill. But I will try to resist degradation of terms that have specific technical meaning.
So I would very much disagree with the statement that memory safety and sandboxing are orthogonal. They are certainly different. Linearly independent even. But with a fair amount of overlap.
But it's much easier to say "orthogonal" than "linearly independent", no?
As you mentioned, I think the word "orthogonal" has already lost its meaning of "dot product equals zero", and bears the meaning of "linearly independent" (i.e. dim(N) > 1) in casual speech.
Sort of similarly, I'd like to see more use of sandboxing in memory-safe language programs. But I don't see a ton of people using these OS primitives in, e.g., Rust or Go.
There's a need for some portable and composable way to do sandboxing.
Library authors you can't configure seccomp themselves, because the allowlist must be coordinated with everything else in the whole process, and there's no established convention for negotiating that.
Seccomp has its own pain points, like being sensitive to libc implementation details and kernel versions/architectures (it's hard to know what syscalls you really need). It can't filter by inputs behind pointers, most notably can't look at any file paths, which is very limiting and needs even more out-of-process setup.
This makes seccomp sandboxing something you add yourself to your application, for your specific deployment environment, not something that's a language built-in or an ecosystem-wide feature.
I think Rust is great for sandboxing because of how Rust has basically no runtime. This is one of the nice things about rust!
Go has the same problems I’m describing in my post. Maybe those folks haven’t done the work to make the Go runtime safe for sandboxing, like what I did for Fil-C.
Sure, but even just setuiding to a restrictive uid or chrooting would go a long way, even in a managed runtime language where syscall restrictions are more challenging.
Can someone give a tldr of what makes fil-c different from just compiling with clang’s address sanitizer?
Calling it memory safe is a bit of a stretch when all it does is convert memory errors to runtime panics, or am I missing something? I mean, that’s still good, just less than I’d expect given the recent hype of fil-c being the savior for making C a competitive language again.
ASan does not make your code memory safe! It is quite good at catching unintentional bugs/oob memory writes in your code, and it is quite reliable (authors claim no false positives), but it has false negatives i.e. won't detect everything. Especially if you're against someone who tries to corrupt your memory intentionally.
ASan works by (simplifying a lot) padding allocations and surrounding them with untouchable "red zone". So with some luck even this can work:
char *a = new char[100];
char *b = new char[1000];
a[500] = 0; // may end up in b
Address sanitizer won’t panic/crash your program on all memory safety violations. Attackers know how to achieve remote code execution in processes running Asan. Asan’s docs specifically call out that you should not use it in prod. In other words, Asan is not memory safe. It’s just a bug finding tool.
Fil-C will panic your program, or give some kind of memory safe outcome (that is of no use to the attacker) in all of the cases that attackers use to achieve remote code execution. In other words, Fil-C is memory safe.
The fact that Fil-C achieves memory safety using runtime checks doesn’t make it any less memory safe. Even rust uses runtime checks (most importantly for array bounds). And, type systems that try to prove safety statically often amount to forcing the programmer to write the checks themselves.
From a definition point of view that might be right and it’s no doubt a good step up, compared to continuing with tainted data. In practice though, that is still not enough, these days we should expect higher degree of confidence from our code before it’s run. Especially with the mountains of code that LLMs will pour over us.
Fil-C introduces a garbage collector and can result in significant slowdowns in some cases. Its main existence reason is making non-perf sensitive C/C++ memory safe, not improving the language design. If really want your stack to be C/C++ & Fil-C then your competition includes D/Nim/Go/etc, not (just) Rust/Zig. Even if it magically made C/C++ memory safe, no downsides, your question basically boils down to C vs C++ vs Rust. Don't know about you but prefer somewhat larger binaries and some compiler brawling over programming 70s style.
Both are great tech but solve the problem of safety differently. I would say Fil-c is great for non-performance-critical (think like somewhere between c and go/java, still very fast) existing C programs where compatibility with the existing program / security is a big concern. think ffmpeg, nginx, sudo.
Fil-c:
- You have a great existing c program that may have memory bugs, and you wanna make it safer.
- Or you wanna write a new program in c, and be extra sure it's safe and don't mind a little performance penalty.
- Or you wanna find subtle memory bugs by building your c program with fil-c (asan style) and disable it for performance in your release build.
Rust is great when you want to build a new codebase from scratch, and have the time and patience to deal with the borrow checker. It also gives you some thread safety, (which is different from memory safety) at the development time cost of dealing with the borrow checker.
Rust:
- A new codebase where you need multithreading and safety, and want excellent performance
- You need a broad ecosystem of existing packages
- Your problem space benefits from a robust type system.
Fil-C aborts your program if it detects unsafe memory operations. You very much can write code that is not memory safe, it will just crash. Also it has significant runtime cost.
Rust tries to prevent you from writing memory-unsafe code. But it has official ways of overcoming these barriers ("unsafe" keyword, which tells compiler - "trust me bro, I know what I'm doing) and some soundness holes. But beause safety is proven statically by compiler, it is mostly zero-cost. ("Mostly" because some things compiler can't prove and people resort to "unsafe" + runtime checks)
Two orthogonal approaches to safety. You could have Fil-C style runtime checks in Rust, in principle.
Another option: It's genuinely easier, for what amounts to namespacing reasons. Like, if I came up with a cool new C compiler, I'd probably name it ${MYNAME}cc just because that's an easy identifier that is very unlikely to have a collision and doesn't require me to spend time thinking of some name that is clever, unique, and accurately conveys what the project is about.
Meh, a lot of people name their projects or product series using puns on their own name, or an abbreviation, not from pride or ego at all but simply for convenience of a unique namespace.
There's a hybrid approach of C -> WASM -> C compilation, which ends up controlling every OS interaction and sandboxing memory access like WASM, while technically remaining C code:
https://rlbox.dev/
WASM sandboxes don't do much to guarantee the soundness of your program. It can hose your memory all it wants, it can just only do so within the confines of the sandbox.
Using a sandbox also limits what you can do with a system. With stuff like SECCOMP you have to methodically define policies for all its interactions. Like you're dealing with two systems. It's very bureaucratic and the reason we do it, is because we don't trust our programs to behave.
With Fil-C you get a different approach. The language and runtime offer a stronger level of assurance your program can only behave, so you can trust it more to have unfettered access to the actual system. You also have the choice to use Fil-C with a sandbox like SECCOMP as described in the blog post, since your Fil-C binaries are just normal executables that can access powerful Linux APIs like prctl. It took Linux twenty years to invent that interface, so you'll probably have to wait ten years to get something comparable from WASI.
> It can hose your memory all it wants, it can just only do so within the confines of the sandbox.
True, although as I understand it the WASI component model at least allows multiple fine-grained sandboxes, so it's somewhere in-between per-object capabilities and one big sandbox for your entire program. I haven't actually used it yet so I might be wrong about that.
> so you'll probably have to wait ten years to get something comparable from WASI
I think for many WASI use cases the capability control would be done by the host program itself, so you don't need OS-level support for it. E.g. with Wasmtime I do
But yeah a standard WASI program can't itself decide to give up capabilities.WASI is basically CORBA, and DCOM, PDO for newer generations.
Or if you prefer the bytecode based evolution of them, RMI and .NET Remoting.
I don't see it going that far.
The WebAssembly development experience on the browser mostly still sucks, especially the debugging part, and on the server it is another yet another bytecode.
Finally, there is hardly any benefit over OS processes, talking over JSON-RPC (aka how REST gets mostly used), GraphQL, gRPC, or plain traditional OS IPC.
> hardly any benefit over OS processes, talking over JSON-RPC
Hardly any benefit except portability and sandboxing, the main reasons WASM exists?
WASM sacrifices guest security & performance in order to provide mediocre host sandboxing, though. It might be a useful tradeoff sometimes, but proper process-based sandboxing is so much stronger and lets the guest also have full security & performance.
How is process-based sandboxing stronger? Also the performance penalty is not only due to sandboxing (I doubt it's even mostly due to it). Likely more significant is the portability.
> How is process-based sandboxing stronger?
Because the guarantees themselves are stronger, process isolation is something we have decades of experience with, it goes wrong every now and then but those are rare instances whereas what amounts to application level isolation is much weaker in terms the guarantees it that it provides and the maturity level of the code. That suggests that if you base your isolation scheme on processes rather than 'just' sandboxing that you will come out ahead and even with all other things the same you'd have one more layer in your stack of swiss cheese slices. A VM would offer another layer of protection on top of that, one with yet stronger guarantees.
process-based sandboxing has hardware support and thus stronger defenses against things like spectre. So far every CPU on the market has only elected to address spectre as it relates to crossing ring or process boundaries. Nobody has added hardware spectre defenses for in-process sandboxing. Also, process-based sandboxing allows the guest to also have a full suite of security protections like ASLR. If you are doing sandboxing for defense in depth, reducing the security of what's inside the guest in turn reduces the security of your entire chain.
And I didn't say the performance penalty was because of sandboxing (although in the case of WASM there is cost as it's doing software enforcement of things that otherwise are "for free" in hardware), but just that WASM has a performance cost compared to native. If you are using WASM just for sandboxing, you still then pay a performance cost for portability you didn't need.
WASM is only portable if the only thing it does is heating up CPU, given that everything else depends on the host.
No because the host can present the same interface on every platform. I do think that WASI is waaay to much "let's just copy POSIX and screw other platforms", but it does work pretty well even on Windows.
You've named half of the weasel security technologies of the last three decades. The nice thing about SECCOMP BPF is it's so difficult to use that you get the comfort of knowing that only a very enlightened person could have written your security policy. Hell is being subjected to restrictions defined by people with less imagination than you.
That's a sandboxing technology but not a memory safety technology.
You can totally achieve weird execution inside the rlbox.
Running ffmpeg compiled for wasm and watching as most codec selections lead to runtime crashes due to invalid memory accesses is fun. But, yeah, it’s runtime safety, so going to wasm as a middle step doesn’t do much.
> Running ffmpeg compiled for wasm and watching as most codec selections lead to runtime crashes due to invalid memory accesses is fun.
For all you know that’s a bug in the wasm port of the codec.
> it’s runtime safety
So is Fil-C
The problem with wasm is that an OOBA in one C allocation in the wasm guest can still give the attacker the power to clobber any memory in the guest. All that’s protected is the host. That’s enough to achieve weird execution.
Hence why I say that wasm is a sandbox. It’s not memory safety.
I’m not disagreeing with anything you said about wasm?
Finally reality is catching up with the WASM sales pitch against other bytecode formats introduced since 1958, regarding security and how great it is over anything else.
Warm was great because it was lightweight and easy to target from any language and create any custom interaction API with the host. That's becoming less true as they bolt on features no one needed (GC) and popularize standardized interfaces that contain the kitchen sink (WASI) but these things can still be treated as optional so it can still be used for much more flexible use cases than java or .net
> features no one needed (GC)
WasmGC is absolutely necessary for languages like Dart, Kotlin, and Java, which are all using it right now successfully.
But I get that if you're compiling C or Rust then it might seem like it isn't useful.
There’s no guarantee the toolchains will support WASM “preview” forever and make the bloat optional, and even if they do you could still end up in an ecosystem where it would be unviable. At some point you’re probably better off just compiling to RISCV and using an emulator library instead.
Fortunately core wasm is simple enough for a single person to implement an interpreter or even compiler for.
Even if the major engines continue to pile on complexity we have a pretty good escape hatch I think.
Since 1958 (UNCOL) there have been more options than only Java or CLR MSIL.
Wasm now supports multiple modules and multiple linear memories per module, so it ought to be quite possible to compile C to Wasm in a way that enforces C's object access rules, much like CHERI if perhaps not Fil-C itself.
You wouldn't be able to get quite as fine-grained. One memory per object is probably horrifically slow. And I don't know about Fil-C, but CHERI at least allows capabilities (pointers with bounds) to overlap and subset each other. I.e. you could allocate an arena and get a capability for that, and then allocate an object inside that arena and get a smaller capability for that, and then get a pointer to a field in that object and get capability just for that field.
Fil-C has like one "linear memory" per object and each capability gives read/write access to the whole object.
But Fil-C has its compiler which does analysis passes for eliding bounds-checks where they are not needed, and I think it could theoretically do a better job at that than a WASM compiler with multi-memories, because C source code could contain more information. Unlike WASM, but like CHERI, every pointer in memory is also tagged, and would lose its pointer status if overwritten by an integer, so it is still more memory-safe in that way.
It has a separate address space for each object? That seems unlikely. Is it not pretty much a software implementation of CHERI?
One would probably just need to define WASM extensions that allow for such subsetting. Performance will probably be competitive with software implementations of CHERI (perhaps with varying levels of hardware acceleration down the road) which isn't that bad.
The multiple linear memory is supported in wasi preview 3? I thought it was not supported as of preview 2.
Some WebAssembly runtimes now do support those parts of the specification.
The author has a knack for generating buzz (and making technically interesting inventions) :)
I'm a little concerned that no one (besides the author?) has checked the implementation to see if reducing the attack surface in one area (memory security) might cause problems in other layers.
For example, Filip mentioned that some setuid programs can be compiled with it, but it also makes changes to ld.so. I pointed this out to the author on Twitter, as it could be problematic. Setuid applications need to be written super-defensively because they can be affected by envars, file descriptors (e.g. there could be funny logical bugs if fd=1/2 is closed for a set-uid app, and then it opens something, and starts using printf(), think about it:), rlimits, and signals. The custom modifications to ld.so likely don't account for this yet?
In other words, these are still teething problems with Fil-C, which will be reviewed and fixed over time. I just want to point out that using it for real-world "infrastructures" might be somewhat risky at this point. We need unix nerds to experiment with.
OTOH, it's probably a good idea to test your codebase with it (provided it compiles, of course) - this phase could uncover some interesting problems (assuming there aren't too many false positives).
I've been doing just that. If there's a way to break fil-c we're gonna find it.
Wishful thinking: Any possible chance that means you might make a Fil-C APE hybrid? It would neatly address the fact that Fil-C already needs all of its dependencies to also use Fil-C.
If you are really concerned you should do this and then report back. Otherwise it is just a mild form of concern trolling.
I checked the the code, reported a bug, and Filip fixed it. Therefore, as I said, I was a little concerned.
Yes, but instead of remarking solely on the fact that the author has a pretty good turnaround time for fixing bugs (I wished all open source projects were that fast) and listens to input belies the tone of your comment, which makes me come away with a negative view of the project, when in fact the evidence points to the opposite.
It's a 'damning with faint praise' thing and I'm not sure to what degree you are aware of it but I don't think it is a fair way to treat the author and the project. HN has enough of a habit of pissing on other people's accomplishments already. Critics have it easy, playwrights put in the hours.
I understand your point, and I have the utmost respect for the author who initiated, implemented, and published this project. It's a fantastic piece of work (I reviewed some part of it) that will very likely play an important role in the future - it's simply too good not to.
At the same time, however, the author seems to be operating on the principle: "If I don't make big claims, no one will notice." The statements about the actual security benefits should be independently verified -this hasn't happened yet, but it probably will, as the project is gaining increasing attention.
> "If I don't make big claims, no one will notice."
I am making big claims because there are big claims to be made.
> he statements about the actual security benefits should be independently verified -this hasn't happened yet
I don't know what this means. Folks other than me have independently verified my claims, just not exhaustively. No memory safe language runtime has been exhaustively verified, save maybe Spark. So you're either saying something that isn't true at all, or that could be said for any memory safe language runtime.
To clarify the position, my concern isn't that the project is bad - it's that security engineering is a two-front war. You have to add new protections (memory safety) without breaking existing contracts (like ld.so behavior).
When a project makes 'big claims' about safety, less technical users might interpret that as 'production ready'. My caution is caused by the fact that modifying the runtime is high-risk territory where regressions can introduce vulns that are distinct from the memory safety issues you are solving.
The goal is to prevent the regression in the first place. I'm looking forward to seeing how the verification matures and rooting for it.
> without breaking existing contracts (like ld.so behavior)
If you think that Fil-C regresses ld.so then get specific. Otherwise what you’re doing is spreading fear, uncertainty, and doubt for no good reason.
Fil-C has always honored the setuid behavior provided by ld.so. There was a bug - since fixed - that the Fil-C runtime called getenv instead of secure_getenv.
> When a project makes 'big claims' about safety, less technical users might interpret that as 'production ready'.
Fil-C is production ready and already has production users.
I would suggest you re-read your comment in a week or so to see if by then you are far enough away from writing it to see how others perceive it. If it wasn't your intention to be negative then maybe it is my non-native English capability that is the cause of this but even upon re-reading it that's how I perceive it.
- You start off with commenting that the author has a knack for self promotion and invention. My impression is that he's putting in a status report for a project that is underway.
- you follow this up with something that you can't possibly know and use that to put the project down, whilst at the same time positioning yourself as a higher grade authority because you are apparently able to see something that others do not, effectively doing that which you accuse the author of: self promotion.
- You then double down on this by showing that it was you who pointed out to the author that there was a bug in the software, which in the normal course of open source development is not usually enough to place yourself morally or technically above the authors.
- You then in your more or less official capacity of established critic warn others to hold off putting this project to the test until 'adults' have reviewed it.
- And then finally you suggest they do it anyway, with your permission this time (and of course now amply warned) with the implicit assumption that problems will turn up (most likely this will be the case) and that you hope 'there won't be too many false positives', strongly suggesting that there might be.
And in your comment prior to this reply you do that once again, making statements that put words in the mouth of the author.
You're right, my tone was off.
It's difficult for me to have a positive opinion of the author when he responds with dismissal and derision to concerns others have raised about Fil-C and memory safety under data races.
The fact is that Fil-C allows capability and pointer writes to tear. That is, when thread 1 writes pointer P2 to a memory location previously holding P1, thread 2 can observe, briefly, the pointer P2 combined with the capability for P1 (or vice versa, the capability for P2 coupled to the pointer bits for P1).
Because thread 2 can observe a mismatch between a pointer and its capability, an attacker controlled index into P2 from thread 2 can access memory of an object other than the one to which P2 points.
The mismatch of pointer and capability breaks memory safety: an attacker can break the abstraction of pointers-as-handles and do nefarious things with pointers viewed instead as locations in RAM.
On one hand, this break is minor and doesn't appear when memory access is correctly synchronized. Fil-C is plenty useful even if this corner case is unsafe.
On the other hand, the Fil-C as author's reaction to discourse about this corner case makes me hesitant to use his system at all. He claims Java has the same problem. It does not. He claims it's not a memory safety violation because thread 1 could previously have seen P1 and its capability and therefore accessed any memory P1's capability allowed. That's correct but irrelevant: thread 2 has P2 and it's paired with the wrong capability. Kaboom.
The guy is technically talented, but he presents himself as Prometheus bringing the fire of memory safety to C-kind. He doesn't acknowledge corner cases like the one I've described. Nor does he acknowledge practical realities like the inevitability of some kind of unsafe escape hatch (e.g. for writing a debugger). He says such things are unnecessary because he's wrapped every system call and added code to enforce his memory model's invariants around it. Okay, is it possible to do that in the context of process_vm_writev?
I hope, sincerely, the author is able to shift perspectives and acknowledge the limitations of his genuinely useful technology. The more he presents it as a panacea, the less I want to use it.
> Because thread 2 can observe a mismatch between a pointer and its capability, an attacker controlled index into P2 from thread 2 can access memory of an object other than the one to which P2 points.
Under Fil-C’s memory safety rules, „the object at which P points” is determined entirely by the capability and nothing else.
You got the capability for P1? You can access P1. That’s all there is to it. And the stores and loads of the capability itself never tear. They are atomic and monotonic (LLVM’s way of saying they follow something like the JMM).
This isn’t a violation of memory safety as most folks working in this space understand it. Memory safety is about preventing the weird execution that happens when an attacker can access all memory, not just the memory they happen to get a capability to.
> He claims Java has the same problem. It does not.
It does: in Java, what object you can access is entirely determined by what objects you got to load from memory, just like in Fil-C.
You’re trying to define „object” in terms of the untrusted intval, which for Fil-C’s execution model is just a glorified index.
Just because the nature of the guarantees doesn’t match your specific expectations does not mean that those guarantees are flawed. All type systems allow incorrect programs to do wrong things. Memory safety isn’t about 100% correctness - it’s about bounding the fallout of incorrect execution to a bounded set of memory.
> That's correct but irrelevant: thread 2 has P2 and it's paired with the wrong capability. Kaboom.
Yes, kaboom. The kaboom you get is a safety panic because a nonadversarial program would have had in bounds pointers and the tear that arises from the race causes an OOB pointer that panics on access. No memory safe language prevents adversarial programs from doing bad things (that’s what sandboxes are for, as TFA elucidates).
But that doesn’t matter. What matters is that someone attacking Fil-C cannot use a UAF or OOBA to access all memory. They can only use it to access whatever objects they happen to have visibility into based on local variables and whatever can be transitively loaded from them by the code being attacked.
That’s memory safety.
> He doesn't acknowledge corner cases like the one I've described.
You know about this case because it’s clearly documented in the Fil-C documentation. You’re just disagreeing with the notion that the pointer’s intval is untrusted and irrelevant to the threat model.
> The kaboom you get is a safety panic
You don't always get a panic. An attacker who can get a program to access an offset he controls relative to P2 can access P1 if P2 is torn such that it's still coupled, at the moment of adversarial access, with P1's capability. That's dangerous if a program has made a control decision based on the pointer bits being P2. IOW, an attacker controlled offset can transform P2 back into P1 and access memory using P1's capability even if program control flow has proceeded as though only P2 were accessible at the moment of adversarial access.
That can definitely enable a "weird execution" in the sense that it can let an attacker make the program follow an execution path that a plain reading of the source code suggests it can't.
Is it a corner case that'll seldom come up in practice? No. Is it a weakening of memory safety relative to what the JVM and Rust provide? Yes.
You are trying to define the problem away with sleigh-of-hand about the pointer "really" being its capability while ignoring that programs make decisions based on pointer identity independent of capability -- because they're C programs and can't even observe these capabilities. The JVM doesn't have this problem, because in the JVM, the pointer is the capability.
It's exactly this refusal to acknowledge limitations that spooks me about your whole system.
> An attacker who can get a program to access an offset he controls relative to P2 can access P1 if P2 is torn such that it's still coupled, at the moment of adversarial access, with P1's capability
Only if the program was written in a way that allowed for legitimate access to P1. You’re articulating this as if P1 was out of thin air; it’s not. It’s the capability you loaded because the program was written in a way that let you have access to it. Like if you wrote a Java program in a way where a shared field F sometimes pointed to object P1. Of course that means loaders of F get to access P1.
> That can definitely enable a "weird execution"
Accessing a non-free object pointed by a pointer you loaded from the heap is not weird.
I get the feeling that you’re not following me on what „weird execution” is. It’s when the attacker can use a bug in one part of the software to control the entire program’s behavior. Your example ain’t that.
> Is it a corner case that'll seldom come up in practice? No. Is it a weakening of memory safety relative to what the JVM and Rust provide? Yes.
I don’t care about whether it’s a corner case.
My point is that there’s no capability model violation and no weird execution in your example.
It’s exactly like what the JVM provides if you think of the intval as just a field selector.
I’m not claiming it’s like what rust provides. Rust has stricter rules that are enforced less strictly (you can and do use the unsafe escape hatch in rust code to an extent that has no equal in Fil-C).
I think his argument is that you can have code this:
And this happens: 1. s->user is initialized to alice 2. Thread 1 sets s->idx to ((alice - bob) / sizeof(...)) and s->user to Bob, but only the intval portion is executed and the capability still points to Alice 3. Thread 2 executes the if, which succeeds, and then gives all privileges to Alice unexpectedly since the bob intval plus the idx points to Alice, while the capability is still for AliceIt does seem a real issue although perhaps not very likely to be present and exploitable.
Seems perhaps fixable by making pointer equality require that capabilities are also equal.
I understand his argument.
Here are the reasons why I don’t buy it:
1. I’m not claiming that Fil-C fixes all security bugs. I’m only claiming that it’s memory safe and I am defining what that means with high precision. As with all definitions of memory safety, it doesn’t catch all things that all people consider to be bad.
2. Your program would crash with a safety panic in the absence of a race. Security bugs are when the program runs fine normally, but is exploitable under adversarial use. Your program crashes normally, and is exploitable under adversarial use.
So not only is it not likely to be present or exploitable, but if you wrote that code then you’d be crashing in Fil-C in whatever tests you ran at your desk or whenever a normal user tried to use your code.
But perhaps point 1 is still the most important: of course you can write code with security bugs in Fil-C, Rust, or Java. Memory safety is just about making a local bug not result in control of arbitrary memory in the whole program. Fil-C achieves that key property here, hence its memory safe.
> I’m only claiming that it’s memory safe and I am defining what that means with high precision
Do you have your definition of memory safety anywhere? Specifically one that's precise enough that if I observe a bug in a C program compiled via Fil-C, I can tell whether this is a Fil-C bug allowing (in your definition) memory unsafety (e.g. I'm pretty sure an out-of-bounds read would be memory unsafety), or if it's considered a non-memory-safety bug that Fil-C isn't trying to prevent (e.g. I'm pretty sure a program that doesn't check for symlinks before overwriting a path is something you're not trying to protect against). I tried skimming your website for such a definition and couldn't find this definition, sorry if I missed it.
I typically see memory safety discussed in the context of Rust, which considers any torn read to be memory-unsafe UB (even for types that don't involve pointers like `[u64; 2]`, such a data race is considered memory-unsafe UB!), but it sounds like you don't agree with that definition.
In my understanding the program can work correctly in normal use.
It is buggy because it fails to check that s->idx is in bounds, but that isn't problem if non-adversarial use of s->idx is in bounds (for example, if the program is a server with an accompanying client and s->idx is always in bounds when coming from the unmodified client).
It is also potentially buggy because it doesn't use atomic pointers despite comcurrent use, but I think non-atomic pointers work reliably on most compiler/arch combinations, so this is commonplace in C code.
A somewhat related issue if that since Fil-C capabilities currently are only at the object level, such an out-of-bounds access can access other parts of the object (e.g. an out-of-bounds access in an array contained in an array element can overwrite other either of the outer array)
It is true though that this doesn't give arbitrary access to any memory, just to the whole object referred to by any capability write that the read may map to, with pointer value checks being unrelated to the accessed object.
If you set the index to `((alice - bob) / sizeof(...))` then that will fail under Fil-C’s rules (unless you get lucky with the torn capability and the capability refers to Alice).
Exactly. I agree that this specific problem is hard to exploit.
> Seems perhaps fixable by making pointer equality require that capabilities are also equal
You'd need 128-bit atomics or something. You'd ruin performance. I think Fil-C is actually making the right engineering tradeoff here.
My point is that the way Pizlo communicates about this issue and others makes me disinclined to trust his system.
- His incorrect claims about the JVM worry me.
- His schtick about how Fil-C is safer than Rust because the latter has the "unsafe" keyword and the former does not is more definitional shenanigans. Both Fil-C and Rust have unsafe code: it's just that in the Fil-C case, only Pizlo gets to write unsafe code and he calls it a runtime.
What other caveats are hiding behind Pizlo's broadly confident but narrowly true assertions?
I really want to like Fil-C. It's good technology and something like it can really improve the baseline level of information security in society. But Pizlo is either going to have to learn to be less grandiose and knock it off with the word games. If he doesn't, he'll be remembered not as the guy who finally fixed C security but merely as an inspiration for the guy who does.
All I’m really hearing is that this guy rubs you the wrong way, so you’re not going to give him the benefit of the doubt that you’d give to others.
I mean, maybe you’re right that his personality will turn everyone off and none of this stuff will ever make it upstream. But that kind of seems like a problem you’re actively trying to create via your discourse.
> Only if the program was written in a way that allowed for legitimate access to P1. You’re articulating this as if P1 was out of thin air; it’s not.
My program:
If the return statement can access P1, disjoint from P2, that's a weird execution for any useful definition of "weird". You can't just define the problem away.Your central claim is that you can take any old C program, compile it with Fil-C, and get a memory-safe C program. Turns out you get memory safety only if you write that C program with Fil-C's memory model and its limits in mind. If someone's going to do that, why not write instead with Rust's memory model in mind and not pay a 4x performance penalty?
> that's a weird execution for any useful definition of "weird".
Weird execution is a term of art in the security biz. This is not that.
Weird execution happens when the attacker can control all of memory, not just objects the victim program rightly loaded from the heap.
> Your central claim is that you can take any old C program, compile it with Fil-C, and get a memory-safe C program.
Yes. Your program is memory safe. You get to access P1 if p pointed at P1.
You don’t get to define what memory safety means in Fil-C. I have defined it here: https://fil-c.org/gimso
Not every memory safe language defines it the same way. Python and JavaScript have a weaker definition since they both have powerful reflection including eval and similar superpowers. Rust has a weaker definition if you consider that you can use `unsafe`. Go has a weaker definition if you consider that tearing in Go leads to actual weird execution (attacker gets to pop the entire Go type system). Java’s definition is most similar to Fil-C’s, but even there you could argue both ways (Java has more unsafe code in its implementation while Fil-C doesn’t have the strict aliasing of Java’s type system).
You can always argue that someone else’s language isn’t memory safe if you allow yourself to define memory safety in a different way. That’s not a super useful line of argumentation, though it is amusing and fun
You may define "memory safety" as you like. I will define "trustworthy system" as one in which the author acknowledges and owns limitations instead of iteratively refining private definitions until the limitations disappear. You can define a mathematical notation in which 2+3=9, but I'm under no obligation to accept it, and I'll take the attempt into consideration when evaluating the credibility of proofs in this strange notation.
Nobody is trying to hide the existence of "eval" or "unsafe". You're making a categorical claim of safety that's true only under a tendentious reading of common English words. Users reading your claims will come away with a mistaken faith in your system's guarantees.
Let us each invest according to our definitions.
> I will define "trustworthy system" as one in which the author acknowledges and owns limitations instead of iteratively refining private definitions until the limitations disappear.
You know about this limitation that you keep going on about because it’s extremely well documented on fil-c.org
[Woman walking on beach at sunset, holding hands with husband]
Voiceover: "Miracurol cures cancer."
[Couple now laughing over dinner with friends]
"Ask your doctor if Miracurol is right for you."
[Same footage continues, voice accelerates]
"In clinical trials, five mice with lymphoma received Miracurol. All five were cured. One exploded. Not tested in humans. Side effects include headache, itchiness, impotence, explosion, and death. Miracurol's cancer-free guarantee applies only to cancers covered under Miracurol's definition of cancer, available at miracurol.org. Manufacturer not responsible for outcomes following improper use. Consult your doctor."
[Couple walking golden retriever, sun flare]
Voiceover: "Miracurol. Because you deserve to live cancer-free."
Patient: "I exploded."
Miracurol: "That's extremely well documented on miracurol.org."
It’s really sad to see your posts on this thread. Fil-C is an incredible achievement and absolutely full of interesting technical details to dig into. I’m not a mod, but as a reader of the site and someone who takes a curious interest in the progress of Fil-C, can you please stop attacking its creator like this. It’s tedious, needlessly rude, and lessens the opportunity for the rest of us to actually learn something from an expert.
Facts are facts and exist independent of who discovers them. If you'd like to learn, the last thing you want to do is stop people poking at contradictions and pressure-testing claims. If Fil-C is really the "incredible achievement" you say it is, it can withstand scrutiny.
Wait, no, you just said
> I will define "trustworthy system" as one in which the author acknowledges and owns limitations
You can't then go on to complain that the author does document the limitations but considers the overall system good. Fil-C, by the definition you just espoused, is a "trustworthy system".
Sorry to intrude on the discussion, but I have a hard time grasping how to produce the behavior mentioned by quotemstr. From what I understand the following program would do it:
Is this program demonstrating the issue? Does this execute under Fil-C's rules without a memory fault? If not, could you provide some pseudocode that causes the described behavior?No, this program doesn’t demonstrate the issue.
You can’t access out of bounds of whatever capability you loaded.
Fil-C lets programs access objects through the wrong pointer under data race. All over the Internet, you've responded to the tearing critique (and I'm not the only one making it) by alternatively 1) asserting that racing code will panic safely on tear, which is factually incorrect, and 2) asserting that a program can access memory only through its loaded capabilities, which is factually correct but a non sequitur for the subject at hand.
You're shredding your credibility for nothing. You can instead just acknowledge Fil-C provides memory safety only for code correctly synchronized under the C memory model. That's still plenty useful and nobody will think less of you for it. They'll think more, honestly.
> asserting that racing code will panic safely on tear, which is factually incorrect
Try it. That’s what happens.
> through its loaded capabilities, which is factually correct but a non sequitur for the subject at hand.
It’s literally the safety property that Fil-C guarantees.
Safety properties provided by languages aren’t about preventing every bad thing that users can imagine. Just because the language does something different than what you expect - even if it allows you to write a program with a security bug - doesn’t mean that the language in question isn’t memory safe.
> You're shredding your credibility for nothing. You can instead just acknowledge Fil-C provides memory safety only for code correctly synchronized under the C memory model.
Fil-C provides memory safety even for incorrectly synchronized code. That safety guarantee is easy to understand and easy to verify: you only get to access the memory of the capability you actually loaded. You’re trying to evade this definition by getting hung up on what the pointer’s intval was, and your PoC uses a pointer comparison to illustrate that. You’re right that the intval is untrusted under Fil-C rules.
I’m not going to downplay the guarantees of my technology just to appease you. Whether or not you find me credible is less important to me than being honest about what Fil-C guarantees.
In https://news.ycombinator.com/item?id=46270657, you write
> If you set the index to `((alice - bob) / sizeof(...))` then that will fail under Fil-C’s rules (unless you get lucky with the torn capability and the capability refers to Alice).
In the comment above, you write, referring to a fault on access through a torn capability
> Try it. That’s what happens.
Your position would be clearer if you could resolve this contradiction. Yes or no: does an access through a pointer with an arbitrary offset under a data race that results in that pointer's capability tearing always fault?
> You’re right that the intval is untrusted under Fil-C rules.
Can Fil-C compile C?
You can't argue, simultaneously,
1) it's the capability, not your "intval", that is the real pointer with respect to execution flow and simultaneously, and
2) that Fil-C compiles normal C in which the "intval" has semantic meaning.
Your argument is that Fil-C is correct with respect to capabilities even if pointers are transiently incorrect under data races. The trouble is that Fil-C programs can't observe these capabilities and can observe pointers, and so make control flow decisions based on these transient incorrect (you call them "untrusted") inputs.
Can you show an actual minimal C program which has this problem? I’m trying to follow along here, but it’s very hard for me to understand the exact scenario you’re talking about.
I think at this point it's reasonable to conclude that quotemstr does not have a legitimate concern until a program demonstrating the issue can be presented.
> Rust has a weaker definition if you consider that you can use `unsafe`
I don't see it. Rust makes the same guarantees regardless of the unsafe keyword. The difference is only that with the unsafe keyword you the programmer are responsible for upholding those guarantees whereas the compiler can check safe Rust.
C is safe by the same logic, then? You can write safe code in anything if you don’t make mistakes.
But the definition is what we're talking about, not whether you make mistakes. Of course it's important that safe Rust is checked by the compiler, but that's crucially not part of how safety is defined.
I would guess that somebody more on the pulse of C's safety efforts could tell you whether they have a definition of memory safety for C or whether they're comfortable with an existing definition from somebody else.
What I mean is, what’s to stop us saying that C upholds all the same guarantees that Rust does and that it’s the programmer that’s responsible for upholding them (just as the programmer is responsible in the case of Rust code marked ‘unsafe’)? This seems like a semantic game to avoid acknowledging that unsafe Rust comes with some of (though not all) of the same risks as C code.
In short, the definitions are not important. What matters are the risks that you do or don’t run. And if your Rust code contains unsafe blocks, you are running risks that you wouldn’t be if you used Fil-C, which has no such escape hatch. (Of course this goes both ways – your Fil-C code is more likely to fail, safely, with a runtime error due to a mistake that Rust would have caught at compile time.)
And do you say that C offers these guarantees ?
Real world C software does not read like software written by people who are in fact upholding those guarantees you say C could equally have. It reads as though they think such a guarantee is a joke or an irrelevance. It's not rare for me to run into people who think C's pointers are just indexing into a massive array of all RAM (or its equivalent on today's systems with virtual addressing), that's not just not in the same ballpark as a safe C program, that's playing a different sport on another continent.
You seem to be suggesting that a language being safe or unsafe is a social contract rather than a technical property of the language.
>And do you say that C offers these guarantees ?
No, that would be silly, and it's an illustration of why it is silly to say that a language guarantees X if it is the programmer who must check that X holds. If we go down that route (which, to repeat, would be silly), then we can make C safe without any technical changes just by adding some language to the standard saying that C programmers are obliged to ensure that their code maintains a certain list of invariants. When you say that "Rust makes the same guarantees regardless of the unsafe keyword", it seems to me that you are doing something equally pointless.
> You seem to be suggesting that a language being safe or unsafe is a social contract rather than a technical property of the language.
Quite some way up this thread pizlonator insists that each programming language defines memory safety differently, quantifying some as "weaker" or "stronger" and giving the example that Rust has the `unsafe` keyword and so that's weaker than Fil-C.
That's what we were discussing when you jumped in with your C hypothetical.
You apparently instead believe in a single universal "safety" and every language is either absolutely safe or unsafe according to foldr for whatever that's worth - but that's not what we were talking about.
No, I just think that Rust is less safe than it would be if it didn’t have the unsafe escape hatch.
I think you’re taking issue with how pizlonator phrased his post rather than addressing the substance of his point that Fil-C does not have the ‘unsafe’ escape hatch and is therefore safer in this respect. Sure, Rust uses a pretty standard definition of memory safety when talking about the desired property of the program, but pizlonator is talking about the definition of memory safety that the Rust compiler actually guarantees that Rust code will meet, which (when you include unsafe-marked code) is a conditional and weaker one.
I still can't agree with weaker. Yes, it's conditional on the unsafe code actually obeying the rules, and on the tooling, but Fil-C has the same situation, there will be bugs in the compiler, indeed in some cases the same bugs because LLVM has plenty of bugs.
Crucially unsafe Rust doesn't have weaker rules, it has the same rules, that's my whole thrust here. The change is that the burden of obeying those rules is on you, the programmer, and that's a heavy burden. But it is very much possible for skilled practitioners to carefully achieve that. It's very similar skill to writing C++ stdlib implementations. Aria's "Pre-pooping your pants" essay is colourful but ultimately it's the same sort of thing strong exception guarantees are made of in C++. We go in eyes open, expecting the worst so that we're pleasantly surprised when it doesn't happen.
It's not practical for humans to write code like this all day, everyday, they make too many mistakes that's the problem in C or C++ - but, seems like it is practical for some skilled people, sometimes, with the benefit of oversight from similarly skilled peers.
If that isn't enough for you I have good news and I have bad news. The bad news is that for general purpose software too bad, we've known since the middle of last century that we can't do better than this. Fil-C isn't magic, nor are Java and C#. At runtime we detect we can no longer assure correct operation and we abort, this might not be OK, but we can't do better.
The good news is that we can do better if we're willing to sacrifice generality. The difficulties all come from the fact that ultimately if we're a general purpose programming language we can be made into a Gödel number and then obliged to perform computations on ourselves and we're toast. But we can invent useful languages which aren't powerful enough to do that. Want to decompress ZIP files? No generality needed. Crop some JPEGs? Not needed. Validate whether these graphs are isomorphic? Ditto.
Rust is a general purpose language, but you might well not need one for your problem, and I say we should prefer not to use a general purpose language when we don't need one.
I don't think it makes sense to compare non-invariant-respecting unsafe blocks to compiler bugs. It would make sense to do so if unsafe blocks were only present in a highly-verified Rust stdlib, but we both know that's not the case.
>Crucially unsafe Rust doesn't have weaker rules, it has the same rules, that's my whole thrust here. The change is that the burden of obeying those rules is on you, the programmer, and that's a heavy burden.
Now we're circling back to my argument about C. The C standards committee could declare that 'unsafe C' (i.e. all C) has these very same rules (which C programmers have the heavy burden of obeying). Would this instantly convert C into a memory safe language? Of course not! It's an empty semantic gesture. Similarly, merely saying "Rust programmers are obliged to respect the following invariants inside unsafe blocks!" does nothing to actually decrease the risks associated with unsafe blocks (leaving aside whatever exhortive success such admonitions might have).
What next, if we accept this logic? Is Perl a language with strict static typing, but "the burden of checking the types falls on you, the programmer"?.
> Would this instantly convert C into a memory safe language?
I actually strongly encourage finishing this thought. Imagine you're WG14 and you're intending to drop this "safe C" bombshell in C29 and think about the implications just for your document.
You "just" need to spell out all these rules for the language itself, Rust only has a handful but you've got pages of these, and then you need to go re-design all your APIs so that instead of basically YOLO† each of the standard library APIs has an explicable set of safety requirements, just as Rust has to do with its relatively small subset of unsafe APIs.
A subset of this work is already underway for WG14 and has been for a few years. You'll need to hurry them along to prepare for your epoch making announcement because they're not expecting to be anywhere close by 2029 but hey, shoot for the stars right?
At the end of this, you've announced a deeply incompatible C version and the benefit is that if your customers can hire people who don't make mistakes and they port all their C to this new version, it has similar properties to if they were to rewrite it in Rust. Don't expect applause, in fact, I'd recommend hiring bodyguards.
† I think people really underestimate how much C relies on this. Remember C provides qsort, an unstable type-erased comparison sort (hopefully an introsort, but in some implementations literally just Hoare's Quicksort which is significantly older than C itself) which has arbitrary Undefined Behaviour if you screwed up your comparison function and yet for all the popular implementations it's still slower than in Rust which doesn't have that UB problem at all.
So now you're documenting specifically for functions like this why "safe C" is both much harder to use and slower in your standards document like it's an achievement, and unlike Karoline Leavitt you're not even getting paid to do this. Maybe you should take up knitting instead?
You’ve kindly finished the thought for me. You now have a “safe” version of C purely by updating the standard and associated documentation. (It’s not actually true that you need to redesign all the APIs in the stdlib. You just need to document the appropriate restrictions on how they may safely be used, just as you would have to do with a Rust function marked ‘unsafe’. It’s trivially possible to write an unsafe-marked Rust function that’s guaranteed to be memory safe if and only if an arbitrary invariant is maintained.)
In reality of course, this is all absurd. No matter how much or how little work it turns out to be, writing reams of standardese and leaving actual C implementations untouched would do nothing to reduce the safety risks associated with C code.
By the way, you’re wasting some energy arguing points that I agree with (e.g. that Rust has a better sorting API than C). My comments here are not anti-Rust. I merely disagree with the claim that Rust code marked ‘unsafe’ is as safe as regular Rust code.
> ... then we can make C safe without any technical changes just by adding some language to the standard saying that C programmers are obliged to ensure that their code maintains a certain list of invariants.
In Rust you can use #![forbid(unsafe_code)] to totally forbid unsafe code in your codebase. Rust also checks for memory safety at compile time, these are strong guarantees that ensure that if the code compiles it is memory safe.
I'm aware of that, but I'm responding to the original claim that "Rust makes the same guarantees regardless of the unsafe keyword" (see https://news.ycombinator.com/item?id=46262774)
Ah. I agree with you. When unsafe is used the borrow checker cannot check for memory safety, the programmer has to provide the guarantees by making sure their code does not violate memory safety, similar to programming in C.
But unsafe Rust is still far better than C because the unsafe keyword is visible and one can grep it and audit the unsafe parts. Idiomatic Rust also requires that the programmer provides comments as to why that part is unsafe.
I think making things more explicit with "unsafe" is an advantage of Rust, but I think "far better" is a bit of an exaggeration. In C you need to audit pointer arithmetic, malloc/free, casts and unons. If you limit pointer arithmetic to a few safe accessor functions and have a documented lifetime rules, this is also relatively simple to do (more difficult than "grep" but not much). Vice versa, if you use a lot of "unsafe" in Rust or in complicated ways, it can also easily become possible to guarantee safety. In contrast to what people seem to believe, the bug does not need to be inside in unsafe block (a logic error outside can cause the UB inside unsafe or a violation of some of Rust's invariants inside unsafe can allow UB outside of unsafe) and can result even from the interaction of unsafe blocks.
The practical memory safety we see in Rust is much more the result of trying hard to avoid memory safety issues and requiring comments for unsafe blocks is part of this culture.
> the bug does not need to be inside in unsafe block
The argument is that while you wouldn't in fact fix the bug by modifying the unsafe code block, the unsafe code block was wrong until you fixed the other code.
For example imagine if a hypothetical typo existed inside RawVec (the implementation details of Vec) causing the growable array to initially believe it has 1 element inside it, not 0 even though no space has been allocated and nothing was stored. That's safe code, and of course the correct fix would be to change it from 1 to 0, easy. But this broken type is arguably broken because the unsafe code would deference a pointer that isn't valid, trying to reach that non-existent value. It would be insane, perhaps even impossible, to modify that code to somehow handle the "We wrote 1 instead of 0" mistake, when you could instead fix the bug - but that is where the theoretical fault lies.
Since you know C and you know Rust:
I'm curious what you make of quotemastr's point about a race causing a mismatch between the pointer's capability and its index. First off, in your estimation can this realistically be exploited to wreak havoc on extant C programs compiled using Fil-C? Second, is such a mismatch able to happen in safe Rust? Third, is such a mismatch able to happen in unsafe Rust?
Edit: clarification to narrow the question even further
I can try.
"Wreak havoc" is a very vague claim. Instinctively the tearing feels like something very difficult to usefully exploit, but, we know historically that the only people who can reliably tell you whether it was difficult are the attackers actually trying to do it. Don't believe the defenders.
AIUI this capability versus value distinction is a Fil-C thing. So, that's not a thing in Rust at all. In Safe Rust the pointer types, which is what we care about here, aren't very interesting because safe Rust can't dereference them, safe Rust is fine with you making a pointer from the word "LAUGHING" (not a pointer to the string, just the literal bytes in ASCII, but treated as a pointer) or from just some random bytes you found in a data file, because it's not allowed to dereference them so, cool, whatever, no harm no foul.
In unsafe Rust we're allowed to dereference valid pointers, but it's our job to ensure we obey that rule about validity, it being our job to obey rules is what "unsafe" means. So, that silly "LAUGHING" pointer isn't valid, it's just pointer-shaped toxic material. Even if, by coincidence, a pointer you have happened to have the same address as that pointer, in both C and Rust it's not OK to just go around dereferencing invalid pointers, they are not offsets into an imaginary huge array of all memory even though some C programmers act like they are.
Ignoring the Fil-C specific capabilities, in Rust the tearing issue is a matter of synchronization, which is something Rust cares about as part of delivering "fearless concurrency". Rust's marker traits Send and Sync are good place to start learning about that. Yes, we could unsafely implement these marker traits in unsafe Rust when we shouldn't, and thus enable what I imagine you'd call havoc.
So, mostly the problem is that your question is (unintentionally) too vague to answer well but I hope I was at least somewhat helpful.
I'm not an expert here but I have to say this feels like a very weak objection.
p points to P1. One thread reads through p. Another thread races with that and mutates p to point to P2. The result is the first thread reads from either P1 or P2 (but no other object).
This seems totally fine and expected to me? If there's a data race on a pointer, you might read one or the other values, but not garbage and not out of bounds. I mean, if it could guarantee a panic that's nice, but that's a bonus, not required for safety.
Posts like the one I made about how to do sandboxing are specifically to make the runtime transparent to folks so that meaningful auditing can happen.
> For example, Filip mentioned that some setuid programs can be compiled with it, but it also makes changes to ld.so. I pointed this out to the author on Twitter, as it could be problematic.
The changes to ld.so are tiny and don’t affect anything interesting to setuid. Basically it’s just one change: teaching the ld.so that the layout of libc is different.
More than a month ago, I fixed a setuid bug where the Fil-C runtime was calling getenv rather than secure_getenv. Now I’m just using secure_getenv.
> In other words, these are still teething problems with Fil-C, which will be reviewed and fixed over time. I just want to point out that using it for real-world "infrastructures" might be somewhat risky at this point. We need unix nerds to experiment with.
There’s some truth to what you’re saying and there’s also some FUD to what you’re saying. Like a perfectly ambiguous mix of truth and FUD. Good job I guess?
Is it FUD? Approximately speaking, all software has bugs. Being an early adopter for security critical things is bound to carry significant risk. It seems like a relevant topic to bring up in this sort of venue for a project of this sort.
It's true. I used to promote high-assurance kernels. They had low odds of coding errors but the specs could be wrong. Many problems Linux et al. solved are essentially spec-level. So, we just apply all of that to the secure designs, right?
Well, those spec issues are usually not documented or new engineers won't know where to find a full list. That means the architecturally-insecure OS's might be more secure in specific areas due to all the investment put into them over time. So, recommending the "higher-security design" might actually lower security.
For techniques like Fil-C, the issues include abstraction gap attacks and implementation problems. For the former, the model of Fil-C might mismatch the legacy code in some ways. (Ex: Ada/C FFI with trampolines.) Also, the interactions between legacy and Fil-C might introduce new bugs because integrations are essentially a new program. This problem did occur in practice in a few, research works.
I haven't reviewed Fil-C. I've forgotten too much C and the author was really clever. It might be hard to prove the absence of bugs in it. However, it might still be very helpful in securing C programs.
It’s like half FUD.
The FUDish part is that the only actual bug bro is referring to got fixed a while ago (and didn’t have to do with ld.so), and the rest is hypothetical
> a perfectly ambiguous mix of truth and FUD
Congrats on Fil-C reaching heisentroll levels!
My trouble with separate categories "memory safety technology" and "sandboxing technology" is that something like WASM execution is both:
* Depending on how WASM is used, one gets safety guarantees. For example, memory is not executable.
* Privileges are reduced as a WASM module interacts with the environment through the WASM runtime and the embedder
Now, when one compiles C to WASM one may well compile things with bugs. A memory access bug in C is still a memory access bug, but its consequences can be limited in WASM execution. Whether fail-stop behavior is guaranteed actually depends on the code the C compiler generates and the runtime (allocation/deallocation, concurrency) it sets up.
So when we enumerate immediately available security options and count WASM as sandboxing, this is not wrong. But WASM being an execution environment, one could do a lot of things, including a way of compiling and executing C that panics when a memory access bug is encountered.
Wasm is just sandboxing.
Say your C program has sensitive information in module A and a memory safety bug in module B. Running that program in wasm won’t prevent the attacker from using the bug in B to get read/write access to the data in A.
In practice what the attacker will really do is use the memory safety bug to achieve weird execution: even without control over the program counter, the fact that a memory safety bug inside the wasm memory gives read write access to all of that memory means the attacker can make the program do whatever they want, subject to the wasm sandbox limits (ie whatever the host allows the wasm guest to do).
Basically wasm amounts to a lightweight and portable replacement for running native code in a sufficiently sandboxed process
Your general point stands - wasm's original goal was mainly sandboxing - but
1. Wasm does provide some amount of memory safety even to compiled C code. For example, the call stack is entirely protected. Also, indirect calls are type-checked, etc.
2. Wasm can provide memory safety if you compile to WasmGC. But, you can't really compile C to that, of course...
Correct me if I'm wrong, but with LLVM on Wasm, I think casting a function pointer to the wrong type will result in you calling some totally unrelated function of the correct type? That sounds like the opposite of safety to me.
I agree about the call stack, and don't know about GC.
That is incorrect about function pointers: The VM does check that you are calling the right function type, and it will trap if the type does not match.
Here it is in the spec:
> The call_indirect instruction calls a function indirectly through an operand indexing into a table that is denoted by a table index and must have type funcref. Since it may contain functions of heterogeneous type, the callee is dynamically checked against the function type indexed by the instruction’s second immediate, and the call is aborted with a trap if it does not match.
From https://www.w3.org/TR/wasm-core-2/#syntax-instr-control
(Other sandboxing approaches, including related ones like asm.js, do other things, some closer to what you mentioned. But wasm has strict checking here.)
Hmm, I wonder if I was confusing emscripten's asm.js approach with its approach to Wasm. Thank you!
Depends on how it is used is already a sign that WebAssembly isn't really as safe as being sold, by many of its advocates, versus other bytecode formats.
Like, C is actually really safe, it only depends on how it is being used.
People only have to enumerate the various ways and tools to write safe code in C.
Problem solved, or so we get to believe.
WASM is just a bytecode format for a stack based vm. Granted it is weirdly named, the actual "Assembly" equivalent is WAT.
But the point is, it is a format specification, which has nothing to do with safety. You can implement a totally unsafe WASM runtime if you so choose. Personally I think it's not a bad thing, at least we have something like it that can run in a browser environment. But I am curious to know why you dislike it so much.
> including a way of compiling and executing C that panics when a memory access bug is encountered.
WASM couldn’t do that because it doesn’t have a sense of the C memory model nor know what is and isn’t safe - that information has long been lost. That kind of protection is precisely what Fil-C is doing.
WASM is memory safe in that you can’t escape the runtime. It’s not memory safe in that you can escape escape the program running within the sandbox, which you can’t do with a memory safe language like Rust or Fil-C.
MicroVMs seem to be getting more popular.
I wonder how they fit into the picture.
Good point!
It would require a bit of porting (since Fil-C currently assumes you have all of the Linux syscalls). But you could probably even lift some of the microVM’s functionality into Fil-C’s memory safe userland.
I hope this project gets more traction. I would love to see a memory safe battle tested sudo or polkit in my package manager without having to install a potentially workflow breaking replacement.
If you're into Nix, check out https://github.com/mbrock/filnix — not yet integrated & maintained in upstream Nixpkgs, but lets you replace Nix/NixOS packages with Fil-C versions quite easily.
I am into nix! it's me and my gf's daily driver. thank you.
Great! Feel free to ask questions or report problems on GitHub or the Fil-C Discord. If some package doesn't compile I can probably look into it!
Why doesn't it mention landlock?
Because that’s not the sandboxing tech that either OpenSSH or the browsers use as far as I can tell.
But landlock works great in Fil-C.
Thanks, I see. From the title I didn't infer that it's browser or openssh specific but that's ok.
Fair point!
I should add that landlock just works, and that lots of the other sandboxing techs also work
because it's shit
By "it" do you mean landlock or fil-c? Also elaboration would be nice.
Ok, if you say so
Which requirements does a full blown virtual machine not meet? By leaning on that as the sandbox, we get Qubes, but maybe I don't know what I'm talking about.
It’s true that a full blown VM is an excellent sandbox.
The usual situation is like what chrome or OpenSSH want:
- They want to be able to do dangerous things by design. Chrome wants to save downloads. Chrome wants to call rendering APIs. OpenSSH wants to pop a shell.
- They want to deal with untrusted inputs. Chrome downloads things off the internet and parses them. OpenSSH has a protocol that it parses.
So you want to split your process into two with privilege separation:
- one process has zero privileges and does the parsing of untrusted inputs.
- another process has high privilege but never deals with untrusted inputs.
And then the two processes have some carefully engineered IPC protocol for talking to one another.
Could you run the deprivileged process in a VM for maximum security? Yeah that’s one way to do it. But it’s cleaner to run it as a normal process, ask the OS to sandbox it (deprivilege it), and then have a local domain socket (or whatever) that the two processes can use to communicate.
If you used a VM for deprivileging then:
- There’d be more overhead. Chrome wants to do this per origin per tab. OpenSSH wants to do it per connection. Maybe a VM is too much
- You could put the whole browser into the VM but then you’d still need something outside it for saving files. And probably for talking to the GPU. You could run OpenSSH in the VM but then that defeats the purpose (you want to use it to pop a shell after all).
- You can use vsocks and other things to communicate between host and guest but it’s much more gross than the options available when using traditional process sandboxing
Does it even work with openssh example? Pwning the parser progress will let attacker spoof arbitrary communication, which in case of SSH lets them execute arbitrary commands. Or is there a smart way to work around that?
You can send arbitrary commands, but they will be rejected unless you provide valid credentials first.
OS-level sandboxes are way too coarse grained to achieve a good "hollowing out" of the attack surface. The principle of least privilege should extend down to/start at the individual language library level (because this is where the actual trust boundaries are), or even finer grained, at the individual function or code segment level (thereby providing maximum control), and therefore not be limited to larger domains.
Most software today relies on many (imported, third party) libraries, so the security architecture should provide primitives/abstractions to manage rights at that level, which requires programming languages to implement the ability to sandbox (managing the effects of) code. If they did this with lightweight, portable virtual machines like WebAssembly, that could work.
The vast majority of code out there should be limited to pure computation and have no ability to access anything external at all (and otherwise, only what it actually requires) - yet most languages are simply incapable of providing any such guarantees. If the programmer of software cannot get ironclad assurances, they cannot in turn provide them to their users.
I'm not saying that OS-level sandboxing isn't good, just that it doesn't go far enough. And depending on the setup, it may not sufficiently limit the effects of compromised elements, and it provides no "monitoring in the small". It's also not convenient or efficient to have an entire OS instance for every single system component. Compartmented microkernel operating systems like Genode do it better imo.
When it comes to VMs, most things are solved and have near native performance, but desktop graphics are not. Due to limitations in GPU architecture, you usually have to dedicate an entire GPU to the VM to have reasonable graphical acceleration. Qubes doesn't solve this either, IIRC the apps running in VMs are glued to the host with X11 forwarding without any acceleration support.
It's planned to allow GPU for chosen, trusted VMs on Qubes: https://github.com/QubesOS/qubes-issues/issues/8552
For what it's worth the Intel B50 supports GPU virtualization and software support seems to be improving quickly. Really hoping this blows the lid off over the next couple years.
Nit：The word “orthogonal” should not mean merely “different”. It should mean “completely unrelated” if we are drawing a proper analogy from linear algebra. Orthogonal vectors have a dot product of zero. No correlation whatsoever. As ML and linear algebra terms spread to more common language of course the terms will change their meaning. Just as “literally” now often means “figuratively” I’m not going to die on this hill. But I will try to resist degradation of terms that have specific technical meaning.
So I would very much disagree with the statement that memory safety and sandboxing are orthogonal. They are certainly different. Linearly independent even. But with a fair amount of overlap.
But it's much easier to say "orthogonal" than "linearly independent", no? As you mentioned, I think the word "orthogonal" has already lost its meaning of "dot product equals zero", and bears the meaning of "linearly independent" (i.e. dim(N) > 1) in casual speech.
Sort of similarly, I'd like to see more use of sandboxing in memory-safe language programs. But I don't see a ton of people using these OS primitives in, e.g., Rust or Go.
There's a need for some portable and composable way to do sandboxing.
Library authors you can't configure seccomp themselves, because the allowlist must be coordinated with everything else in the whole process, and there's no established convention for negotiating that.
Seccomp has its own pain points, like being sensitive to libc implementation details and kernel versions/architectures (it's hard to know what syscalls you really need). It can't filter by inputs behind pointers, most notably can't look at any file paths, which is very limiting and needs even more out-of-process setup.
This makes seccomp sandboxing something you add yourself to your application, for your specific deployment environment, not something that's a language built-in or an ecosystem-wide feature.
To be properly useful as a sandbox, it would be nice to have a tool that would run another process/executable in a sandboxed environment.
Basically a tool that would allow to run flatpaks/AppImages/ etc...
Maybe firejail already does all that can be done without using a VM.
I think Rust is great for sandboxing because of how Rust has basically no runtime. This is one of the nice things about rust!
Go has the same problems I’m describing in my post. Maybe those folks haven’t done the work to make the Go runtime safe for sandboxing, like what I did for Fil-C.
Sure, but even just setuiding to a restrictive uid or chrooting would go a long way, even in a managed runtime language where syscall restrictions are more challenging.
Some of the properties of fil-c managed heaps are very similar to what CHERI can do with Cornucopia by the way: see https://dl.acm.org/doi/10.1145/3620665.3640416
Just dropping this: I can't load the link from Spain, as La Liga blocks access to it - football censoring.
Can someone give a tldr of what makes fil-c different from just compiling with clang’s address sanitizer?
Calling it memory safe is a bit of a stretch when all it does is convert memory errors to runtime panics, or am I missing something? I mean, that’s still good, just less than I’d expect given the recent hype of fil-c being the savior for making C a competitive language again.
ASan does not make your code memory safe! It is quite good at catching unintentional bugs/oob memory writes in your code, and it is quite reliable (authors claim no false positives), but it has false negatives i.e. won't detect everything. Especially if you're against someone who tries to corrupt your memory intentionally.
ASan works by (simplifying a lot) padding allocations and surrounding them with untouchable "red zone". So with some luck even this can work:
Address sanitizer won’t panic/crash your program on all memory safety violations. Attackers know how to achieve remote code execution in processes running Asan. Asan’s docs specifically call out that you should not use it in prod. In other words, Asan is not memory safe. It’s just a bug finding tool.
Fil-C will panic your program, or give some kind of memory safe outcome (that is of no use to the attacker) in all of the cases that attackers use to achieve remote code execution. In other words, Fil-C is memory safe.
The fact that Fil-C achieves memory safety using runtime checks doesn’t make it any less memory safe. Even rust uses runtime checks (most importantly for array bounds). And, type systems that try to prove safety statically often amount to forcing the programmer to write the checks themselves.
If you can rely on memory errors panicing before the memory error can have an effect, you're memory safe. Memory safety doesn't require "can't crash".
Exactly. Or Rust wouldn't be memory safe due to the existence of unwrap().
Not that crashing can't be bad, as we saw recently with Cloudflare's recent unwrap-based incident.
Even without unwrap, Rust could still crash on array out of bounds access. And probably more similar cases.
From a definition point of view that might be right and it’s no doubt a good step up, compared to continuing with tainted data. In practice though, that is still not enough, these days we should expect higher degree of confidence from our code before it’s run. Especially with the mountains of code that LLMs will pour over us.
It's a nice ambition, but it's a different thing than memory safety
Any wizard here? Fill-C vs Rust? Realistic comments without hype.
Fil-C introduces a garbage collector and can result in significant slowdowns in some cases. Its main existence reason is making non-perf sensitive C/C++ memory safe, not improving the language design. If really want your stack to be C/C++ & Fil-C then your competition includes D/Nim/Go/etc, not (just) Rust/Zig. Even if it magically made C/C++ memory safe, no downsides, your question basically boils down to C vs C++ vs Rust. Don't know about you but prefer somewhat larger binaries and some compiler brawling over programming 70s style.
One thing that I am interested in is the performance of Fil-C compared to other compiled programming languages that are also GC'd, especially Go.
Both are great tech but solve the problem of safety differently. I would say Fil-c is great for non-performance-critical (think like somewhere between c and go/java, still very fast) existing C programs where compatibility with the existing program / security is a big concern. think ffmpeg, nginx, sudo.
Fil-c:
- You have a great existing c program that may have memory bugs, and you wanna make it safer.
- Or you wanna write a new program in c, and be extra sure it's safe and don't mind a little performance penalty.
- Or you wanna find subtle memory bugs by building your c program with fil-c (asan style) and disable it for performance in your release build.
Rust is great when you want to build a new codebase from scratch, and have the time and patience to deal with the borrow checker. It also gives you some thread safety, (which is different from memory safety) at the development time cost of dealing with the borrow checker. Rust:
- A new codebase where you need multithreading and safety, and want excellent performance
- You need a broad ecosystem of existing packages
- Your problem space benefits from a robust type system.
FYI nginx and Apache and Postgres using shared memory across processes make them hard to use with Fil-C. Lighttpd works well though!
Fil-C aborts your program if it detects unsafe memory operations. You very much can write code that is not memory safe, it will just crash. Also it has significant runtime cost.
Rust tries to prevent you from writing memory-unsafe code. But it has official ways of overcoming these barriers ("unsafe" keyword, which tells compiler - "trust me bro, I know what I'm doing) and some soundness holes. But beause safety is proven statically by compiler, it is mostly zero-cost. ("Mostly" because some things compiler can't prove and people resort to "unsafe" + runtime checks)
Two orthogonal approaches to safety. You could have Fil-C style runtime checks in Rust, in principle.
[dead]
What's the rationale for naming it after yourself?
Another option: It's genuinely easier, for what amounts to namespacing reasons. Like, if I came up with a cool new C compiler, I'd probably name it ${MYNAME}cc just because that's an easy identifier that is very unlikely to have a collision and doesn't require me to spend time thinking of some name that is clever, unique, and accurately conveys what the project is about.
Cough Linux cough
You're getting cool tech for free. You can use it or not use it. Complaining about the name is weird.
Linus didn’t name Linux, someone else named it after him.
He did name git after himself though :)
I guess you're kidding, but just in case: no he didn't. The "git" was Larry McVoy.
Throughout history, the prospect of glory has motivated people to do great deeds. Nothing wrong with it now.
Meh, a lot of people name their projects or product series using puns on their own name, or an abbreviation, not from pride or ego at all but simply for convenience of a unique namespace.
Pride is as good a reason as any other