On the contrary, fully cured silicone is virtually impossible to remove from aluminum. It has the curious property of adhering tenaciously, and yet not sealing out water. Even when you think you have it all off, there is often the thinnest of films left behind.
The caulk referred to as "self leveling" was likely Flow Seal, by Dyco. Boy does it flow. I had it flow through the mandrel of standard (not Olympic) rivets, and out the bottom!
Vulkem can be removed, but it is tedius. You remove all you can mechanically (scraper, beveled edge craft sticks, whatever), and then use mineral spirits to soften and remove the rest.