Travel has been downright dangerous throughout most of human history. The world (particularly wilderness areas such as mountains) is full of predatory animals who are not smart enough to realize on an instinctive, species-wide level that those animals (i.e. humans or other sentient beings) have, in evolutionarily recent times, become much more dangerous to hunt. Climbing mountains risks life and limb from falls, low temperatures and altitude sickness, and is simply very hard to do at all, let alone safely.
Even if someone idly tries to climb over a mountain range (and doesn't turn back to get home in time for dinner), very few people can afford to leave their occupations to do so without the promise of a tangible reward at the other side: If no-one knows what is over the mountains, and when they get there they find that it is nothing, it could be a disaster.