By providing at least the wattage which your speakers are designed to handle, you will get a fuller, unstrained sound. And, it would seem you're not satisfied with the volume ... so, you're probably driving your head unit to clipping ... which can DAMAGE your speakers! Yes, boys and girls, underpowering your speakers will damage them when you keep pushing your amp to distortion. Get the amp, you won't regret it.
Your amp's wattage should be at least equal to your speaker's rated power handling capability. I would say more, actually. Better to have some head room, so your amp isn't even breathing hard. Then, exercise some rerstraint to not overpower the speakers.
And, yeah, it's a good idea to cut the volume back when you arrive home, to avoid irritating the neighbors.