This is partly why only aphantasics argue for the "it is only semantics" explanation. For anyone else, this explanation makes no sense, because they have at least one vivid example showing otherwise. Only if you have no vivid visual experiences does this argument have any appeal.
Also the well-documented cases of acquired aphantasia are especially troubling for the skeptical "semantic disagreement" cases, given that acquired aphantasia is entirely within-person.