I assume you're trying to remove the center window, not the wing windows, correct?
To remove the center window, you'll need to disconnect the lifter mechanism, and then lift the window up past 90 degrees, and it will fall out of the channel. Your lifting mechanism is different from my 74, so hopefully someone else will be able to jump in who is more familiar with it.
The rivets you pointed out look like they're attaching the lifting mechanism to the window frame. It looks like there might be screws you could use to remove the lifter without drilling out rivets. How is the lifter arm attached to the window itself?
If you are trying to remove the wing windows, you're in for a much more involved project. You can see our process for removing, refurbishing, and replacing our wing windows on our 1974 Overlander in our blog here:
Wing Windows
Sikaflex 221 is great for sealing between glass and aluminum, rubber and aluminum, and aluminum and aluminum. I'd get as much old sealant out as possible and fill the gaps with Sikaflex.