Hacker News
new
past
comments
ask
show
jobs
points
by
quietFalcon
9 hours ago
|
comments
by
snehesht
9 hours ago
|
next
[-]
They have some community benchmarks published
https://github.com/Niko1221/Strata/tree/main/bench/results
reply
by
merbanan
9 hours ago
|
prev
|
[-]
Q2_0 does 33 tok/s decode and ~600t/s prompt processing at 128k context on RTX2060 8GB VRAM.
ISTA IQ3_XXS does ~21 tok/s decode and ~240t/s prompt processing
reply