It might be conceptually similar to a single-output-token LLM (sort of). LLMs output next-token probabilities. You can ask LLMs to output yes/no, or to output only a color, or only a digit or something like that.

In this case I would imagine that they probably embed your input data into a vector space, and they embed your questions/outputs into another space, and manage to predict probabilities/classes/scores for your outputs very quickly. Embedding the output classes/questions into a vector spaces gives you something you can reuse across runs cheaply, as opposed to an LLM where you can prefill the KV cache but this is an expensive operation in terms of memory.

Basically it's: skip decode, just do prefill then do some measurements. That's the crude description anyways.

And prefill is way faster on GPU type hardware.