Throughput is not a problem as you just share relatively small vectors (a few kilobytes in size), the key issue is network latency.

That isn’t true. llama RPC is incredibly slow but staged splits in skippy are orders of magnitude faster.