I have 32GB of RAM with 16GB VRAM and I haven't had a lot of luck running larger models like this. Are you able to expand on that?
replyuse llama.cpp with cuda
replyThe problem may be that it's a 7800XT which handles memory contention by freezing.
reply