I would expect such a language would have a different effect than you anticipate.
Making a typical language which exists in the gaps between words is spectrally inefficient. If the words are not important, then all that matters is the starts and stops (and maybe a final tone to each word). If all of the information is conveyed between the stops, it would be inefficient to waste your time on the words themselves. You'd almost instantly start dropping words, because they had no value in the language.
The end result of this would look similar to Morse code. While the content of Morse code is actually in the spaces, we hear the punctuation between them. Each word would be similar to the punctuation at the end of each dot or dash in Morse. Having words would be the equivalent to every single press of a Morse code key being a spoken nonsense word.
There's all sorts of issues there. Morse code depends very much on maintaining a clear even timing. If you were interjecting words, you may drop timing while speaking, making it harder to distinguish the value of the space after it. Morse makes it easy, ever click is just that... a click. There's also bandwidth efficiency. Morse code is slightly more efficient than texting. Can you imagine how slow a culture would communicate if they were limited to texting bandwidth?
Now if we start to explore atypical parts of language instead, we find some interesting results. It is very clear that, in English, the spacing between words matters. People intentionally pause at different places to transmit meaning. Thus we know some information gets transmitted that way, but what information is it? Sure, it might just be "the speaker has chosen to pause, because you can hear the background noise," but sometimes you swear the way your mother pauses lets you know whether you can play for another 5 minutes or get your butt inside!
One might imagine a language for a people who are very in tune with their environment. Speaking is an efficient form of communication, but what if all you needed to do was modulate the sound around you? What if you had enough of a sense as to what environment noises were out there to weave a story using them. There are wonderful stories of pianists coming up behind a child who is pecking away at Twinkle Twinkle Little Star, and improvising around them to make the child's song seem magical.
In this sense, their language would be that of a gate. They either let the sound pass (by being silent), or squelch it (by making noise), depending on if that was the sound they wanted to have the listener hear. In theory this is just Morse code all over again, but there's a twist. What if every single squelch changed the sounds around them. If we speak, the birds in the trees shut up, for fear of a predator. What if their speaking changed what the birds chose to say near them. Now they could rely on the birds to speak for them, and merely cultivate their nearby birds to tell the story they want to tell.
Sound crazy? Perhaps. But it strongly resembles how the human brain operates. Many of the most visible effects we see in the human brain are inhibition effects, squelching signals. In fact, many systems which need to amplify a signal do so by inhibiting an inhibitor (a double negative, in effect). Maybe there's something to it after all.
