This was the experiment itself https://github.com/xaskasdf/ps2-llm
replyThe idea was basically to run a llm on a ps2, then I ran into some problems as the 32mb ram cap with 4mb vram cap; so I had to figure out a way to stream layers on the forward pass. Given that ps2 manages to give instructions directly to the vram that's capable of 32bit addresses, it gave an insane amount of tok/s, then I wondered if I could do the same on my puter
I wonder too, DMA plays a huge role in most older gaming consoles when the CPUs were far more sluggish.
replyPerhaps that's what made them think to try.
Perhaps the current batch of smart memory cards which on the PS2 I believe have quite complex DMA capabilities to stream from the SD card game data.
Why not the PS5? That's when games started streaming assets straight from the NVME SSD to the GPU. In this case the assets are weights.
replyActually I'm thinking about buyin an AMD BC-250 that's bassically a ps5 with pcie factor format; and it's linux capable by default, maybe next month
replyJust because he mentioned retro gaming.
replyOtherwise DMA is everywhere.
In the PS5 case since it uses unified memory it's not quite the same as say an GBA streamed from a flash cart to video RAM.