At the end of the day, you either engage with the speech and argue against the idea, or you appeal to authority and hope it is worthy of your trust. Personally, I'm a little too jaded for that.
It's not always simple; it's not always perfect; but practically we do it all the time as a matter of course. Everyone makes decisions like that every day; every authority makes those decisions - managers at work, politicians in hearings; the public does it listening to political messages, parents do it listening to teenagers explaining where they were all night ...