When pulling the window out of my cabin door, I used plastic shims, available at any big box store, or maybe even your local hardware place. Get one started, tapping gently, and then continue around, adding shims as you go. Slow, even pressure will do the job in time without tearing anything up.