Get a nail punch and sharpen it to a fine point.
After punching the rivet dead center, use a 1/8 drill bit first, assuring you stay in the center.
Apply as little pressure as needed.
Next use a 5/32 bit using little pressure to avoid the shavings getting trapped between the metal.
The gold color bits work best for aluminum.
Use air pressure to blow out any remaining shavings. Fit the new olympic rivet prior to putting a dap of vulkem in the hole, in case you have to slightly enlarge the hole.
Once you gain experience, you can skip the 1/8" bit.