Welcome to the forums.
Yes, you can remove the windows to clean them, all it takes is a ratchet and socket(either 1/2" or 7/16"). Unlatch the window from inside, lift the window, unbolt the support arms from the window frame, and lift the lower part of the window way up past 90 degrees. The hinge will drop out of the lip holding it in place. You can then clean the windows, or the area inside the windows, and replace the screens. Installation is the reverse of removal, just make sure the hinge is all the way inserted into the lip before lowering the window.
The blue rubrail and beltline inserts are available from Airstream, or Inland RV.
Meddle not in the affairs of dragons, for you are crunchy, and taste good with ketchup.