There is nothng like a rivet shaver to shave Olympic rivets. Everything else either takes forever, delivers questionable results, or just plain does not work, or in some cases I have seen, does more damage than good.
The rivet shaver, once set up, makes Olympic rivets look just like bucked rivets, and does not damage the skin surrounding the rivet.
I have set quite a few of these Olympic rivets, and shaved them, eventually with a rivet shaver. Comparing the makeshift method to the rivet shaver, the shaver always wins, hands down.
The purchase price, although steep, pales in comparison on the money you can save in repairs. One trip to a RV shop for metal repair can cost you a multiple of the price of the shaver.