The same 24GB VRAM RTX 4090 I bought to play Cyberpunk 2077 with.
replyWorks perfectly fine in llama.cpp throwing 70+t/s at me with 128k q8 K/V context when using the IQ4_NL quant + MTP at q4 MTP K/V.
Also leaving this here because you might find it useful: https://hypfer.github.io/will-it-fit-llama-cpp/
Nice! Do you do anything with that compute when you're not actively using it? Is the crypto-mining hobby still worth it? I've also wondered if such expensive hardware can be rented back out to offset cost. Looks like these cards are going for as much as $4k nowadays.
replyThere are services where you can hook your card up and rent it out to other users. I don't know what any of them are called, but they do exist.
replySalad.com is one. (I’m unaffiliated, just happened to come across it this week while looking for a cheap option)
replyI've paid ~2k€ in 2023. Since I'm usually sitting next to it, I'm only using it when I want to use it. It can get quite loud and warm.
replyCrypto (to my knowledge at least) moved away from GPU mining. I guess you could maybe rent out GPU compute, but - being in germany - it's not worth the legal hassle. You could of course always commit tax fraud, though I wouldn't recommend that.
> I've also wondered if such expensive hardware can be rented back out to offset cost.
replyMassive legal liability. Not worth it.
What did you call me?
reply