Have you considered, since you can extract the images via the mask, selectively inverting them?
One can fairly reliably use a small NN to classify images by whether they should be inverted or just dimmed, and I've used it with great success for years now on my site: https://invertornot.com/https://gwern.net/invertornot
---
On a side note, it'd be nice to have an API or something to let one 'compile' a PDF to dark-mode version PDF. Ephemeral browser-based is a drawback as often as a benefit.
That's actually exactly where I started. The initial idea involved a YOLO nano model to classify images, deciding what to invert and what not to. It worked as a concept, but during the feasibility analysis I realized that for native PDFs it wasn't necessary: the format already tells you where the images are. I walk the page's operator list via getOperatorList() (PDF.js public API, no fork) and reconstruct the CTM stack, that is the save, restore and transform operations, until I hit a paintImageXObject. The current transformation matrix gives me the exact bounds. I copy those pixels from a clean render onto an overlay canvas with no filters, and the images stay intact. It's just arithmetic on transformation matrices, on a typical page it takes a few milliseconds.
Your approach with a classifier makes a lot more sense for the generic web, where you're dealing with arbitrary <img> tags with no structural metadata, and there you have no choice but to look at what's inside. PDFs are a more favorable problem.
A case where a classifier like yours would be an interesting complement is purely vector diagrams, drawn with PDF path operators, not raster images. Veil inverts those along with the text because from the format's perspective they're indistinguishable. In practice they're rare enough that the per-page toggle handles them, but it's the honest limitation of the approach.
> In practice they're rare enough that the per-page toggle handles them, but it's the honest limitation of the approach.
I don't understand how you handle raster images. You simply cannot invert them blindly. So it sounds like you just bite the bullet of never inverting raster images, and accepting that you false-positive some vector-based diagrams? I don't see how that can justify your conclusion "it wasn't necessary". It sounds necessary to me.
Actually, raster images are never inverted, they're protected. The CSS filter: invert() hits the entire canvas (text and images together), then the overlay paints the original image pixels back on top, restoring them. The result is: inverted text, images with their original colors.
The choice to never invert raster images isn't a compromise, it's the design decision. The problem veil solves is exactly that: every dark mode reader today inverts everything, and the result on photos, histology, color charts, scans is unusable. Preserving all images is the conservative choice, and for my target (people reading scientific papers, medical reports, technical manuals) it's the right one.
It's absolutely true that there's a subset of raster images, like diagrams with white backgrounds and black lines, that would benefit from inversion. I could be wrong, but in my experience they're a minority, and the cost of accidentally inverting the wrong one (a medical photo, a color chart) is much higher than the benefit of inverting a black and white diagram, from my point of view. For now the per-page toggle covers those cases.
That really means a lot, ainch. I hope it makes your late-night sessions a little more bearable. If you find anything that doesn't work well with the papers you read, keep me posted
Have you considered, since you can extract the images via the mask, selectively inverting them?
One can fairly reliably use a small NN to classify images by whether they should be inverted or just dimmed, and I've used it with great success for years now on my site: https://invertornot.com/ https://gwern.net/invertornot
---
On a side note, it'd be nice to have an API or something to let one 'compile' a PDF to dark-mode version PDF. Ephemeral browser-based is a drawback as often as a benefit.
That's actually exactly where I started. The initial idea involved a YOLO nano model to classify images, deciding what to invert and what not to. It worked as a concept, but during the feasibility analysis I realized that for native PDFs it wasn't necessary: the format already tells you where the images are. I walk the page's operator list via getOperatorList() (PDF.js public API, no fork) and reconstruct the CTM stack, that is the save, restore and transform operations, until I hit a paintImageXObject. The current transformation matrix gives me the exact bounds. I copy those pixels from a clean render onto an overlay canvas with no filters, and the images stay intact. It's just arithmetic on transformation matrices, on a typical page it takes a few milliseconds.
Your approach with a classifier makes a lot more sense for the generic web, where you're dealing with arbitrary <img> tags with no structural metadata, and there you have no choice but to look at what's inside. PDFs are a more favorable problem.
A case where a classifier like yours would be an interesting complement is purely vector diagrams, drawn with PDF path operators, not raster images. Veil inverts those along with the text because from the format's perspective they're indistinguishable. In practice they're rare enough that the per-page toggle handles them, but it's the honest limitation of the approach.
> In practice they're rare enough that the per-page toggle handles them, but it's the honest limitation of the approach.
I don't understand how you handle raster images. You simply cannot invert them blindly. So it sounds like you just bite the bullet of never inverting raster images, and accepting that you false-positive some vector-based diagrams? I don't see how that can justify your conclusion "it wasn't necessary". It sounds necessary to me.
Actually, raster images are never inverted, they're protected. The CSS filter: invert() hits the entire canvas (text and images together), then the overlay paints the original image pixels back on top, restoring them. The result is: inverted text, images with their original colors.
The choice to never invert raster images isn't a compromise, it's the design decision. The problem veil solves is exactly that: every dark mode reader today inverts everything, and the result on photos, histology, color charts, scans is unusable. Preserving all images is the conservative choice, and for my target (people reading scientific papers, medical reports, technical manuals) it's the right one.
It's absolutely true that there's a subset of raster images, like diagrams with white backgrounds and black lines, that would benefit from inversion. I could be wrong, but in my experience they're a minority, and the cost of accidentally inverting the wrong one (a medical photo, a color chart) is much higher than the benefit of inverting a black and white diagram, from my point of view. For now the per-page toggle covers those cases.
As a PhD student doing my fair share of midnight paper-reading I think I'm the exact target market - thank you for sharing!
That really means a lot, ainch. I hope it makes your late-night sessions a little more bearable. If you find anything that doesn't work well with the papers you read, keep me posted