The "milky look is fading of the clearcoat. Once gone the areas that get the most sun will need to be polished out some. How much depends on you. The toolbox polish seems to work well, use mineral spirits first, wipe it on, then polish, and then some mineral spirits to wipe off after you get a lot of black going. It takes good gloves and a lot of rags.
How shiny you want to go, will gauge how hard you have to work at it. I strippped the vinyl off my door ( inside part) and have been polishing it out, its taken fine steel wool and a lot of elbow grease to get it shiny, and not yet there.