The First Amendment doesn't protect the speaker against all forms of defamation (though it does put some barriers up that make it harder to win in some circumstances). If it did, defamation as a cause of action wouldn't exist at all.
As a practical matter, though, this is largely theoretical. Once you've been through the rigamarole of arrest, prosecution, and trial, even if you're found not guilty of the crimes committed, the reputational damage is just too widespread. You're not going to go after the defamers: there are just too many, and if you tried, there would be a fair question as to whether you have any positive reputation left to injure. Your life is pretty much ruined. It's a pretty terrible situation for the wrongly accused.