The neurons here are MLP neurons, with activations directly after the nonlinearity, so there is definitely a distinguished basis.