There are two ways I can see which would wreck the ability for humans to hear speech:
This is along the same lines as John Robinson's answer, but on a smaller scale. Rather than a lot of wind, the wind would only have to be chaotic enough to destroy sound waves.
Rather than areas without wind, you would have periods of time without wind, though they would be extremely brief (ie, every other microsecond). This could possibly be justified by the presence of microscopic, ferrous particles suspended in the atmosphere all over the planet. The dust could vibrate as a result of fluctuations in the planet's magnetic field (some process occurring in the world's core, unique to this planet?). This would create a constant buzz or hum which would disrupt all spoken language.
Active Noise Cancellation:
The same concept as ANC headphones-- incoming sound waves are duplicated, but 180 degrees out phase. The two contradictory waves meet, and cancel one another out.
In the case of an entire world, this could be managed with nanoscale machines (or the ferrous particles being manipulated by a controlled magnetic field) spread throughout the atmosphere, either intentionally or as a remnant of some ancient technology.
Alternatively, there could be microscopic flora or fauna throughout the atmosphere which actively counteracts any sound waves which would disturb their position as they float. Since every individual microorganism is counteracting the sound which it encounters, this wouldn't just cancel human speech, but all sound below a certain volume.