How Will Federated Voting Work?

Surely there is no way to prevent vote manipulation if you allow votes from any instance to be counted. How does lemmy plan to get lots of vote information to help sort posts on small instances without being vulnerable to manipulation?