Still, if you had the authorities to announce a product of large primes - whatever that means - how do you know they announced the correct primes and not something they made up? I'm afraid any clever cryptographic scheme you come up with falls the victim of the same dilemma: if you can check that your vote was counted right, you lose anonymity. Also how do you check no fake votes were added?
As far as I know, the Estonian voting system works in two stages. First there is an authenticator, that verifies that each given vote is signed properly, is only given once and can be counted, without looking at the payload. Then it anonymizes the votes and signs the result file which is transferred to another system which counts the votes. The communication is unidirectional so that the front-end doesn't get any feedback from the back end save some data transfer acks.
As I said, a decent job. It's not perfect in any meaning of the word. It's practical and it works. People trust it because they trust the people who designed it and run it.
My take is that one should be able to check that a blockchain has not been altered/substituted, and the way to do that is by being able to record encrypted running tally blockchain heads, but this gives you no indication that the published blockchain faithfully represents the cast votes so you have to add paper ballots and an immediate partial manual audit of the paper ballots checked against the digital cast vote record.