The visors in my car had little clip protrusions that snapped into the edges of the holes in the header trim. I had to give 'em one helluva yank to get 'em to pop out...
FWIW, I did the dastardly deed today; finally got the visors out & dropped only one side of the header. I think I may just have been using the wrong swear-words yesterday...
I may have found what bulged out on your header trim, as I got mine to re-hang nice and flat against the windshield: I gave the header a little extra tug to re-seat the two hooks that point rearward into their holes in the header steel, and everything went back together neat as you please.
It may be because I only dropped one side of the header trim at a time - I dropped the passenger side just barely enough to run the RJ11 cable across to the center of the car, over the dome light, then re-seated it - I skipped pulling the A-pillar trim, and just stuffed the cable behind it by pulling the door weatherstrip. I also didn't unhook the passenger side of the header trim from the header; I just lowered the back edge enough to stuff the cable in there...
Then I reassembled the passenger-side of the header trim, and dropped the driver's side completely, as the cable's RJ11 plug emerges from the header just to the left of the center dome-light bulge, and I had to drop the forward edge of the trim to run the cable through. But I was able to re-hang it completely flat. About the only thing that gave me any grief there was re-seating the little tab on the header trim that snaps into the top of the A-pillar trim.
Anyway, it's all in place now, and looks pretty darn sexy, if I do say so myself. Thanks for the hints; it's appreciated. I'll never understand why my visors were such a PITA, though...