Yea... it annoys me, too. especially in warmer weather, the top of the window seems to stick quite well to the seal. Which is also probably the reason why the small rubber piece at the top of the window between the front and rear glass is kinda torn at the top of the front glass.... as i'd assume yours probably is, too.
I usually just take a scrap cloth and rub the crap out of it. Takes some effort, but it comes off. Would probably be easier if i used Windex or something... but eh.