Simple answer to your "either-or" question is: Yes
As others have stated, hunter gatherers were tied to food sources. Germany's prehistoric climate would have been vastly different than it is today, but suffice it to say that hunter-gatherers thrived in this region.
As you stated, for most of the hunter-gatherer period, most of mankind lived in caves or other natural shelters. What this implies (and supported by science) is that "societies" of hunter-gatherers would be made up of small, dispersed groups. These groups are often thought to always be at odds with others, but quite a bit of evidence points that there was a lot of intermixing. Also, trade was actually a factor even during this time, meaning our hunter-gatherer ancestors had it in their best interests to be friendly when possible. (don't forget, they were every bit as intelligent as we are today)
To answer your question, the best evidence I can provide is in the above link concerning a similar climate to prehistoric Germany. The Lake Baikal region has a lot of scientific literature available from this time period, but suffice it to say, there was intermixing (on a society basis at least) as various family-groups moved around between areas. The evidence points towards some groups moving and some staying.
Why this is can be complicated to answer definitively, but it can also be answered simply, at least generally: Those who stayed saw benefit in staying, those who moved saw benefit in moving. Some would have been right, some would have been wrong. But if an area can't support a large group, some are going to have to move or fight for the limited food/shelter resources. It seems most humans chose to voluntarily break off and move to find greener pastures while some would stay and keep working the existing known locations.
So basically, small family groups formed the very loose 'society' and would move around as necessary, not necessarily on a seasonal basis. Moving means burning energy, burning energy means having to find more food. As technology caught up (fire, shelter-building, better tools, eventually agriculture), their dietary budgets became such where taking risks could be allowed and true society started flourishing.