upvote

points

by 9999000009992 days ago |
comments
upvoteby jeroenhd2 days ago|
next
[-]
Nvidia is opening their source code because they moved most of their source code to the binary blob they're loading. That's why they never made an open source Nvidia driver for Pascal or earlier, where the hardware wasn't set up to use their giant binary blobs.

It's like running Windows in a VM and calling it an open source Windows system. The bootstrapping code is all open, but the code that's actually being executed is hidden away.

Intel has the same problem AMD has: everything is written for CUDA or other brand-specific APIs. Everything needs wrappers and workarounds to run before you can even start to compare performance.

reply
upvoteby ycombinator_acc1 days ago|
parent|
next
[-]
Huawei’s CANN is fully open and supposedly a drop-in replacement for CUDA. The latter could make it a supеrior option to either AMD or Intel.
reply
upvoteby Asmod4n2 days ago|
parent|
prev|
[-]
In the python eco system you can just replace CUDA with DirectML in at least one popular framework and it just runs. You are limited to windows then though.
reply
upvoteby lrvick2 days ago|
prev|
next
[-]
Nvidia has been pledging that for years. If it ever actually happens, I am here for it.
reply
upvoteby shaklee32 days ago|
parent|
[-]
It happened 2 years ago:

https://developer.nvidia.com/blog/nvidia-transitions-fully-t...

reply
upvoteby cyberax2 days ago|
parent|
[-]
Their userspace is still closed. ROCm is fully open.
reply
upvoteby pjmlp2 days ago|
parent|
[-]
Provided you happen to have one of those few supported GPUs.

Thus being open source isn't of much help without it.

reply
upvoteby cmxch2 days ago|
prev|
[-]
> Intel

For some workloads, the Arc Pro B70 actually does reasonably well when cached.

With some reasonable bring-up, it also seems to be more usable versus the 32gb R9700.

reply
upvoteby MrDrMcCoy2 days ago|
parent|
[-]
I have both of those cards. Llama.cpp with SYCL has thus far refused to work for me, and Vulkan is pretty slow. Hoping that some fixes come down the pipe for SYCL, because I have plenty of power for local models (on paper).
reply
upvoteby cmxch1 days ago|
parent|
[-]
Hmm.

I had to rebuild llama.cpp from source with the SYCL and CPU specific backends.

Started with a barebones Ubuntu Server 24 LTS install, used the HWE kernel, pulled in the Intel dependencies for hardware support/oneapi/libze, then built llama.cpp with the Intel compiler (icx?) for the SYCL and NATIVE backends (CPU specific support).

In short, built it based mostly on the Intel instructions.

reply