But I thought Nvidia “loves open source” (they don’t) and they are now a supporter for open source and open weight models by acquiring Huggingface? (They don’t actually care)
But the line is drawn when it involves CUDA and any part of their closed source compilers (nvcc).
There are obvious reasons why they are closed source, but it’s becoming pointless since Deepseek have open sourced their AI compiler and compute libraries with DeepGEMM and eventually they will catch up.
I'm rather surprised that it doesn't take the z-buffer as an input. I would have thought that would have provided useful information, it's one of the more useful forms of contolnet.
I think this is mainly so it can use the existing hooks for DLSS upscaling without requiring changes to the renderer, AMD is working on a comparable method which uses adapter networks to slot normals and material properties from the renderer into the diffusion model: https://gpuopen.com/learn/temporally-stable-generative-illum...
The official one seems to do, as well as other info from the engine (I think remember their mentioning LOD/UV map hints in one of the public demos, or articles, a few months back--or it might have been an Unreal Engine podcast)
Even relatively small RGB -> depth models are pretty good. Which kind of implies depth is well encoded in the RGB, and adding depth would not really reduce entropy, while costing bandwidth.
Yes, see the numbers below. In most games, it's basically unusable if you want to play on 60 FPS or above unless you have a 5090.
The current implementation is more of a tech demo than a practical way to play games (+ officially it's available in 1 game). It's _fast enough_ to make some impressive YouTube videos but you most likely won't want to play anything with it yet.
Nvidia has stated that they're still working on improving the performance. No doubt future hardware generations will also include further hardware optimisations.
The potential for this kind of technology is pretty awesome, especially given that people have also found ways to add this to emulators.
Modders have added the ability to use the upscaler after DLSS 5, personally I don't know how sound this method is but the quality is pretty good, and allows to play games with DLSS 5 at 4k 60FPS with something that is not a 5090.
Worth remembering it is running a single-step diffusion model working in pixel space to generate each frame, it's a technical feat in itself that people are even using the words "frames per second"
I assume this is meant to run with the weights people extracted from the latest NBA game, where it was first trialled.
> Isn't the mote that Nvidia has is they work with studios to generate the training data from the game, then they ship a model per game?
That was true for the very first version of DLSS, from DLSS 2 on the models have been universal - the per-game adjustments are done on the inference end by changing the effect intensity or masking out objects
They used to ship one model per game but now there is a single model, however they still do minor updates to it presumably to fine-tune it on new games
Am I the only one who feels a sense of disinterest in a project where the main README is LLM-generated? Does the author not have time to write what they did and how it's used?
I'm more upset about it being factually wrong, e.g. both mentions of "git-ignored" are absurd (why would you mention it if it's not in the repo?) and wrong (they are in the repo).
I notice this, that AI likes to write about things that are not in there. Like i review AI generated output, notice unnecessary things, and asks AI to remove that. So AI removes that and adds that "this and that, that was used or described like this, was removed because bla bla bla" to the document.
I think its somehow needs to talk (write) about the things that are in the context and removal is there so AI predicts that it should be there.
Yeah, I call it bugfix storytelling. Once upon a time this class far far away had this red hooded method...
Especially egregious if both adding and removing the thing happens in one commit. Git should be telling the story, and if it can't then there _is_ no story!
"Am I the only one who feels a sense of disinterest in a project where the code is LLM-generated? Does the author not have time to code the project?"
This is how I feel about every single project announcement on HN recently, they are already bragging about models all over the place, why shouldn't they go full way down being replaced by the Borg?
If the README is >90% AI generated and it is as long as a novel, I am not going to read it and will assume that the author did not read or write it either.
Unfortunately it is slop, beyond the comprehension of the author unless they are experienced with DLSS internals to explain it in depth.
Seems quite similar to this repository?
https://github.com/aloshdenny/open-dlss
Same exact commits at the same time as well, but different repository names and authors. What the hell?
bit-exact against the original )
just unsure who's original ))
Jensen : Nobody needs to code anymore...
Programmer: OpenDLSS...
Jensen : Wait. Not like that! (╯°□°）╯︵┻━┻
But I thought Nvidia “loves open source” (they don’t) and they are now a supporter for open source and open weight models by acquiring Huggingface? (They don’t actually care)
But the line is drawn when it involves CUDA and any part of their closed source compilers (nvcc).
There are obvious reasons why they are closed source, but it’s becoming pointless since Deepseek have open sourced their AI compiler and compute libraries with DeepGEMM and eventually they will catch up.
They only care because open-weight helps to drive the GPU business notably thanks to inference providers (Baseten, Together, Mistral, ...).
At least their support helps the open-weight ecosystem.
Depends on which open source you are talking about, like every single company contributing to FOSS.
They care when the agendas align, and they don't when they won't.
I'm rather surprised that it doesn't take the z-buffer as an input. I would have thought that would have provided useful information, it's one of the more useful forms of contolnet.
I think this is mainly so it can use the existing hooks for DLSS upscaling without requiring changes to the renderer, AMD is working on a comparable method which uses adapter networks to slot normals and material properties from the renderer into the diffusion model: https://gpuopen.com/learn/temporally-stable-generative-illum...
The official one seems to do, as well as other info from the engine (I think remember their mentioning LOD/UV map hints in one of the public demos, or articles, a few months back--or it might have been an Unreal Engine podcast)
Even relatively small RGB -> depth models are pretty good. Which kind of implies depth is well encoded in the RGB, and adding depth would not really reduce entropy, while costing bandwidth.
> bit-exact against the original
What kind of sorcery is this ? Very impressive work !
The load-bearing kind.
LLMs are really good at deobfuscating or even decompiling code.
Almost 8ms on 1080p resolution seems extremely expensive, does the original also eat into the rendering budget as much?
Yes, see the numbers below. In most games, it's basically unusable if you want to play on 60 FPS or above unless you have a 5090.
The current implementation is more of a tech demo than a practical way to play games (+ officially it's available in 1 game). It's _fast enough_ to make some impressive YouTube videos but you most likely won't want to play anything with it yet.
Nvidia has stated that they're still working on improving the performance. No doubt future hardware generations will also include further hardware optimisations.
The potential for this kind of technology is pretty awesome, especially given that people have also found ways to add this to emulators.
Modders have added the ability to use the upscaler after DLSS 5, personally I don't know how sound this method is but the quality is pretty good, and allows to play games with DLSS 5 at 4k 60FPS with something that is not a 5090.
Yes the original is very expensive. It depends on the resolution and the GPU used:
RTX 5060: 9.9 ms at 1080p
RTX 5070: 10.2 ms at 1440p
RTX 5080: 13.7 ms at 2160p
RTX 5090: 8.2 ms at 2160p
Source: https://www.youtube.com/watch?v=3EfLjmdG29Q&t=600
Worth remembering it is running a single-step diffusion model working in pixel space to generate each frame, it's a technical feat in itself that people are even using the words "frames per second"
The original does tend to reduce the FPS by half or more
How useful is this without weights?
Isn't the mote that Nvidia has is they work with studios to generate the training data from the game, then they ship a model per game?
Or is my knowledge outdated here and they're just using a single generalised model?
I assume this is meant to run with the weights people extracted from the latest NBA game, where it was first trialled.
> Isn't the mote that Nvidia has is they work with studios to generate the training data from the game, then they ship a model per game?
That was true for the very first version of DLSS, from DLSS 2 on the models have been universal - the per-game adjustments are done on the inference end by changing the effect intensity or masking out objects
They have a technical report on the neural rendering part of DLSS 5 which goes into it: https://research.nvidia.com/labs/adlr/files/DLSS5_Report.pdf
They used to ship one model per game but now there is a single model, however they still do minor updates to it presumably to fine-tune it on new games
> they ship a model per game?
there's no way that's true!?
>they ship a model per game?
They don't. Only DLSS 1 was trained specifically per each game.
Am I the only one who feels a sense of disinterest in a project where the main README is LLM-generated? Does the author not have time to write what they did and how it's used?
I'm more upset about it being factually wrong, e.g. both mentions of "git-ignored" are absurd (why would you mention it if it's not in the repo?) and wrong (they are in the repo).
I notice this, that AI likes to write about things that are not in there. Like i review AI generated output, notice unnecessary things, and asks AI to remove that. So AI removes that and adds that "this and that, that was used or described like this, was removed because bla bla bla" to the document.
I think its somehow needs to talk (write) about the things that are in the context and removal is there so AI predicts that it should be there.
AI writing is just bad, this things is really noticable - but even in READMEs they look superficially OK until you read them.
AI probably should not be writing docs, commit logs or comments.
Yeah, I call it bugfix storytelling. Once upon a time this class far far away had this red hooded method...
Especially egregious if both adding and removing the thing happens in one commit. Git should be telling the story, and if it can't then there _is_ no story!
If you think about it, actually the author did write what they did (nothing), and also how it’s used (it isn’t).
Seems like the owner of the github repo claims copyright, though. Since they provide a license.
Slop is a new language and you will learn it read it
"Am I the only one who feels a sense of disinterest in a project where the code is LLM-generated? Does the author not have time to code the project?"
This is how I feel about every single project announcement on HN recently, they are already bragging about models all over the place, why shouldn't they go full way down being replaced by the Borg?
I'm fine with it
If the README is >90% AI generated and it is as long as a novel, I am not going to read it and will assume that the author did not read or write it either.
Unfortunately it is slop, beyond the comprehension of the author unless they are experienced with DLSS internals to explain it in depth.