Ours does it with the top closed. It seems to be a recent occurrence, and I'm thinking that the top material is stretching out as the car breaks in. It has a little over 2,300 miles on it. It's most noticeable in the scenario Padawan mentions - windy days and big trucks.
We haven't brought the car back to the dealer yet, but this will certainly be brought up when we do - along with the "dome" light that sometimes stays on all night, and the plastic panel squeak on the driver side quarter, where we got whacked by a minivan 912 miles after we bought the car.