Use Permetex Urethane caulk. Comes in a blue and orange tube from Home Depot.
Sounds crazy, but it cleans with mineral spirits, is white, sticks like stink. Not sure about plexiglass.
Go to my blog, grandbankschoices, and see the process. I did 22? windows. It will never need to be done again...