Better keep the KV cache in full precision
replyWow.. the GOAT himself.. thank you sooo much for creating llama.cpp ... will re-deploy with full kv cache once requests stop coming.
replyI genuinely love talking to these models
replyhttps://ofo1j9j6qh20a8-80.proxy.runpod.net/#/chat/5554e479-0...
I'm contemplating whether I should drive or walk to the car wash (I just thought of that one HN post) and this is what it said after a few back-and-forths:
- Drive to the car (5 minutes), then park and wash.
- If you have a car wash nearby, you can walk there (2 minutes) and do the washing before driving to your car.
- If you're in a car wash location, drive to it and wash there.
Technically the last point was fine, but I like the creativity.
Update: this has been evicted by runpod as it was on spot.
replyThank you! I am impressed by the speed of it.
replyThat was really impressive. https://pastebin.com/PmJmTLJN pretty much instantly. (Very weak models can't do this.)
replyKind sir, May I say to you thanks for doing so! I really appreciate it :D
reply[dead]
reply