Please don't take this as gospel, as I can only relate my own experience with the late sixties Philips/Cornings. I've studied and, contemplated their workings and conclude that although much maligned in the past, because there were few or no rebuild parts available, today there are parts, and their clarity and simplicity makes them a relatively inexpensive and easy restore.
Keep in mind that no matter how carefully and accurately your window rebuild is, they are still going to leak. I have six windows, and only one succumbs a few ounces in a wind-driven rain. I did not employ the silicone injection shop-fix that some do. You'll eventually find that explanation using the google window of the Airforum search function. There's more than enough info about them archived at this site.
Pardon the long preface to answering your question. Yes you can clean them, or better yet, VTS, a supplier that re-manufactures Airstream parts at a fair and reasonable price, sells the entire mechanism for twenty bucks. Every piece of the reproduction mechanism is directly interchangeable with the original. VTS did an exemplary job with this reproduction. I rebuilt all of mine, and only bought three reproductions for their guts, so as to preserve and match the original patina and Philips logo on the covers.
Guessing from your photos it appears to me that your window might be a late '66 type II. That's a two piece glass bar, and much less nightmare than type I when it comes to glass replacement. The year to year transition from type I to type III is somewhat fuzzy, and perhaps differed by supply to California or Ohio assembly factories.
You should seriously consider re-taping the glass into the glass bar, as the original foam tape turns to dust and your priceless chemically tempered glass falls out unexpectedly. Sure, there's “replacement” glass, but it's not the same. It is best to have a helper if you remove the glass.
There is no agreement as to the best rubber perimeter seal, flat or hollow “D”, but I don't think it matters, 'cause that's not the primary area of leakage. They mostly leak through the glass bar hing and at its edges. Inland Andy explains it well. If all your windows look like the one that you show, I'd plan on four weeks of work to fully restore all.
Don't ever feel like a damned fool or question your sanity for getting mixed up with a '66 Overlander. It's one of the best ever Airstreams. I'm five years into fixin' my '68GT and still happy as a pig in …...