They phrase it in a confusing way, with a lot of fluff. But to my understanding what happens is 1. At some part in the transformer, calculate the Jacobian (matrix gradients for that forward pass). Do this for 1000 prompts and average them. 2. For another pass take an "activation vector" (the input vector x to that part). Apply the matrix in step 1 to this vector. 3. Unembed the output to get a token. Presumably to make this have meaning, they use the reading of past tokens after embedding, instead of a random vector in the middle of the transformer. The high level idea appears to be you're trying to find which token, if the value were changed slightly, creates the largest downstream change.

It's not the same thing as the output logits because activations in the J-space are observable even when the model is told explicitly to not speak about a set of inputs. For example, when asked

> Compute 3^2 - 2 while writing "The old painting hung crookedly on the wall"

The model will output only "The old painting hung crookedly on the wall" (and the output logits will reflect that), but activations for "9" and "7" are observable in the J-space.