People would immediately figure out that by using certain reasons, their downvote is removed. And then they would stop giving those reasons. They would then start giving reasons that look legitimate, but who is going to judge whether a reason is "legitimate"? Definitely not the save owner, they shouldn't have that power (and they would always remove downvotes). Moderators could, but that sounds like an incredible amount of work to moderate every single downvote reason. We just can't do that.

In addition, I think the list of users that vote on your save is very private information, save authors should never be able to see the usernames.