Not only is soldering your wires to your speaker a more secure connection, it also provides a better signal flow. Think of it a surface area. If the wires are clipped to a connector, the the connector is only touching the outside strands of the wire. The signal from the inside wires had to flow to the outside wires to get into the clip. Then, the clip isn't touching the entire surface of the speaker terminal either. If it's a tight connection, it will only be touching the sides, and a little of the front and back.
Now, if you replace that with solder, you fill in all the space with a conductive substance and provide a place for all the signal to flow. The wires are all saturated with solder so that the solder is connecting every single strand, instead of just the outside strands. The solder also comes into contact with a larger surface area on the terminal too. So, solder allows a greater amount of signal to flow more freely to your speaker leads (which are soldered to the terminals for the same reason BTW), and gives you better results.
Add in the fact that solder isn't going to rattle free like a clip would, and you have a slam dunk argument. Speaker clips are cheap and easy, and don't require much work, but they are not the right way to make that connection. I would go out and buy an iron and some solder and do it up proper! Just remember the hotter the iron (more wattage) the easier and better the connection will be.