Think about the practical aspect of it. I have to point my phone at you to record you. It's really quite conspicuous. It's also mildly inconvenient for me so I won't be doing it the vast majority of the time.

Whereas the glasses point wherever you're looking, are expected to be recording constantly, and are expected to do things with the data involving third parties. It's the same as a VR headset except in that case the expectation is that the footage is neither sent anywhere nor even retained, merely presented live to the user as if he were looking at you (and his face is already point in your direction).