Happy to answer questions!
edit: now do the 4090 ;)
4000s is WIP, forks has already unlocked it but I'm doing some refactoring to better support different families and after that I'll port it over :-)
If you can't wait for 4090 someone has a fork with a seemingly working unlock :D
I'm running 8 GPUs attached to a single CPU and that driver is a pretty essential part of getting this to work with P2P between as many cards as possible. The modded 4090 is a problem because the BAR1 it advertises (32G) is smaller than the VRAM so not all software is able to use it (you need two memory segments rather than a single 48G one to make this work). That driver is part of the solution for this problem.
Heh, got it working, thank you so much!!
Overall 30% reduction in wall clock time, that's pretty neat (and I'm still not running this at full power but 450 W is a bit much for that puny connector).
Now i'm curious - am I missing some settings? Do I need to check any of these BAR things?
But I would suspect China has figured something out given the reports of them buying 5090s in droves and seemingly modifying them.
I suspect that modding a 5090 in such a way that you can sell it is a lot harder than doing the same with a 4090.
edit: seems you have provided some clarity that even if, unlikely soon unless China has and does the world a favor
https://github.com/aikitoria/open-gpu-kernel-modules
The hardware supports it, but then Nvidia disabled it if the driver detects cheaper cards. This bypasses the NVidia slowdown.
https://github.com/keylase/nvidia-patch
So yeah, buy an Nvidia card then download and run all the software needed to make it work to its full potential so Jensen can buy his 500 millionth leather jacket.
- you will need to run your card hotter than you might be comfortable with
- the real pay-off (on some cards, apparently, see downthread) requires increasing the maximum power to 450W
- This is well above what the stock cooler was designed for so a bit more cooling would help
What's nice is that you can stack this mod on top of (or rather, below of, this one loads first) the p2p mod that has been out for a while, they don't seem to interfere with each other.
Obviously doing more compute requires more power but what it does is shift the card down on the frequency curve to keep the wattage in check. So you're right that at stock power limits it hits a cap where the full issuing rate isn't reached but it depends heavily on the exact workload how much juice is left.
The frequency curve also takes heat into account so in general you end up with a nice perf boost for a slightly hotter card.
I've been running this on my 2 3090s for a month now with no issues but they are also water cooled :-)
My EVGA came with stock firmware that started at a 420W and allowed up 500W, of course 500W did nothing for gaming
Can you explain this one? On my GPU I didn't have to change the power limit. Obviously the card might spend more time at max power usage, but it's still within the limit.
Obviously, if there are 3090's out there that respond differently then I will be happy to retract that.
- get the largest PSU you can find (1500W at a minimum, more is better)
- get the ADT PLX Quad PCIe backplane. Run 4 thin (single slot) GPUs inside your main enclosure, stick the ADT interface board in between two of the thin boards (don't stack them close to each other or they won't be able to breathe).
- ensure you have as much or more RAM as you do VRAM
- Ensure your CPU has enough PCIe lanes to run at least five boards in x16 (four for the GPUs in the main enclosure, another for the PLX PCIe switch)
- In the BIOS set BAR1 to 'resizable' and > 4G decoding should be enabled.
- ditch the cables that come with the ADT board, they may work but there are much better ones out there and they're not expensive, this will save you days of troubleshooting
- Build your own enclosure rather than trying to adapt a standard one, it won't work anyway (the board pitch is wrong, as is the placement of the mounting holes).
- Put another four double or triple boards on the backplane.
- Add another 3x 140 mm fan to the front of the case so you get good airflow without the feeling of being next to a Harrier at take-off
- I used 3D printed parts to mount the PSU and the board as well as the fans (see below).
- Make sure the board is properly grounded to the enclosure
- If you're running 3090's or other cards that are P2P capable that were hobbled by NV then look at this https://github.com/aikitoria/open-gpu-kernel-modules driver module.
- Power on the ADT board, wait three seconds then power on the main machine. Enjoy your VRAM :)
If you have more questions please email me and I'll be happy to help out.
https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...
> Unfortunately for Nvidia the booter is vulnerable to a classic buffer overflow that writes straight into the stack which gives us the ability to execute code via Return Oriented Programming (ROP).
Repo for RTX 4090 and 4070 Ti Super: https://github.com/JustAResearcher/gpu-unlock-ada
This unlock is super useful as it improves the GPU performance on computation-bounded neural network workloads like LLM prefilling, image/video generation, and general neural network training, in a way of disabling deliberate software segmentation on the same hardware imposed by Nvidia.
I'm on rtx 50 series, and no free performance uplifting for now. Waiting for more reverse engineering efforts to reveal whether the unlock is possible for 50 series.
For now, I think reverse engineering efforts on 50 series gpus are just beginning. There are projects like reading sensor temperatures directly from the gpu itself instead of using deliberately DOWNGRADED software like nvidia-smi. However, I don't really see these efforts spreading, and people behind them really needs more recognition. Props to these people, too.
However, it required a few extra steps, as the repo is set up for Linux kernel 6.x and I'm on 5.15. Claude Code was able to get me past the hurdles, including code modifications and startup scripts.
Right now, I'm clearly power limited at 170 watts and I have no desire to brick my card or set it on fire.
This is a benchmark test with Qwen 27B Int4 with 4x3090 and using full batching on VLLM to squeeze every bit of the GPU's.
In this case it's a DMA buffer overflow that writes to the stack which in turn can be exploited with Return Oriented Programming. So no unsigned code is ever executed but we still have code execution control.
Certain (mostly AI) workloads. Depends on how hot you're comfortable running your card. The real-world improvement is considerably less due to other things being bottlenecks.
On the other hand, this only applies to a limited set of workloads (training is often memory bandwidth bound for example).
I also wonder how this affects power consumption.
If its memory bandwidth bound you need to raise your batch sizes to leverage more parallelism.
And also it does work on most 3000 series cards as they all run the same GA102 booter that's vulnerable :-) There are some reports of cards with weird vbios layouts having issues but i'll try to get that fixed
Also this one for 4070s shows that its for fp8 as well:
https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...
There's at least 3080 Ti, 3070 Ti, 3070, 3060