Fantasy tends to have several races that are shorter than humans. With Dwarves, the size is explainable by the fact that they're usually depicted as subterranean-dwelling, and height would be a disadvantage there, but what about races like goblins, halflings, hobbits, etc.? What would provide them an advantage significant enough to allow the small size trait to become fairly universal among the race, assuming that they live above-ground?
You don't need an advantage, hominids start very short (~3.5 ft tall), you just need height to not be an advantage.
That said if you need them to get shorter island living organisms are often much smaller than their mainland living relatives, shortages of food, water, and space give smaller animals an advantage in surviving bad times. Island mammoths (pygmy mammoths) for instance are only as tall as humans. Resource scarcity in general can favor smaller sizes. now once they discover how to make boats they can leave the island but they will be stuck with the small size for a long time.
For climbers size can be a double edged sword as well, especially in tree dwellers, the bigger you are the fewer branches will hold you, there is a reason chimps have not gotten any bigger while humans have.
We don't see lisland living humans getting short now because having boats removes the constraint, ahd the vast majority of humans whom have ever lived in islands got their by boat. Butthere were early hominids trapped on islands by rising sea levels.
This is an actual term in biology and describes the decrese in median body size of a population confined to an island in comparison to a population of the same species on the mainland.
The mechanics aren't fully understood, but the availability (or scarcity) of resources like food and territory size are said to play into this. The 'breaking point' of a population is reached much earlier and smaller individuals with their reduced demand on resources have an evolutionary advantage.
Limitations of height advantages
On the other hand, taller hominids have the advantage of looking further than smaller ones to scout out resources and detect predators. Of course this only works in areas with clear lines of sight, like on a savannah. Place said population in a densely forested area and that advantage of height would be limited.
So, how to keep them short?
As said in another answer and together with this one, a rather isolated, densely forested closed region (tropical region for example) would be a good choice.
"Closed Region" could mean an island, but any structure which limits the area where the people can roam would suffice. A volcano or impact crater with limited access might be another good idea.
Are there actual real world examples?
The examples found in another answer - native people in rainforest regions, especially South East Asia - are one such example, and there is the Flores Human, which does fit the whole bill (isolated population in forested region), though the jury is still out there if it is to be regarded as a separate human species or if they had been 'just' homo sapiens with genetic defects.
When it is hot, smaller is cooler.
In environments where ambient temperature are close to body temperatures, shedding excess heat becomes a big problem for exothermic animals. The linked Scientific American article discusses this in the context of hairlessness.
The largest terrestrial mammals—namely, elephants, rhinoceroses and hippopotamuses—also evolved naked skin because they are at constant risk of overheating. The larger an animal is, the less surface area it has relative to overall body mass and the harder it is for the creature to rid its body of excess heat. (On the flip side, mice and other small animals, which have a high surface-to-volume ratio, often struggle to retain sufficient heat.) During the Pleistocene epoch, which spans the time between two million and 10,000 years ago, the mammoths and other relatives of modern elephants and rhinoceroses were “woolly” because they lived in cold environments, and external insulation helped them conserve body heat and lower their food intake. But all of today's megaherbivores live in sweltering conditions, where a fur coat would be deadly for beasts of such immense proportions.
Human hairlessness is not an evolutionary adaptation to living underground or in the water—the popular embrace of the so-called aquatic ape hypothesis notwithstanding [see box on page 26]. Neither is it the result of large body size. But our bare skin is related to staying cool, as our superior sweating abilities suggest.
Megaherbivores were stuck with their large size for other evolutionary reasons. But in hot Africa, in addition to hairlessness some human types also evolved small body size: the pygmies and the Khoi-San. Consider that heat must largely be shed at the interface between body and environment. As size increases, volume increases faster than surface area and the surface area:volume ratio decreases. It is harder to shed heat when you are big.
So too your small humanoids: they evolved in very hot climates and smaller is cooler. The converse is true - if you want to shed heat slowly is is better to be big than small, and larger body sizes are favored.
One way to achieve a small hominid race is to put them in a protein poor region.
Native people in rain forests, the Kalahari Desert and much of South East Asia until recently, range from being shorter than average to almost child size. Vietnamese and Indonesian males are on average 5.3ft (162cm), male Bushmen of the Kalahari Desert are an average of 5ft (152.4cm), and throughout the tropics traditional native people under 5ft are common, with the men of some tribes averaging 4.11ft (125.27cm). When a larger diet with more protein is introduced the average height (and weight) tends to increase significantly.
So for a hominid race, having them evolve in a habitat with limited or poor protein (bugs, some fish, small animals, etc) but lots of carbs (rice, roots, grains, fruits, etc) to keep them healthy, without building up a lot of bone and muscle mass, would naturally be on the small size. If they're in a thick forest, jungle or swamp, being small would also help them crawl under the branches and vines, and climb trees where a larger body would get stuck or break the branches.
For a fantasy world, with fantasy races, any mystical explanation or creation myth will do. You don't even need to bother explaining that, if it is not important for your story setting... Many fantasy worlds such as GURPS Fantasy, Discworld, Neil Gaiman's Sandman stories, the Harry Potter series etc. have elves, gnomes and goblins of different sizes and shapes, and not one bothers to explain the reasons why they are shorter or taller or thinner or whatever.
Outside of fantasy, the average size for adults of a given species has to do with the evolutionary pressures that species has had to face in its history. There are many good reasons already in other answers, such as insular dwarfism in Anonymous's answer.
One reason I can think of, to add to the list, is that in a Fantasy setting you may expect to see a lot of epic combat, and long range warfare - facing magic, arrows, and magic arrows - is bound to be commonplace. Little people make for smaller targets. That makes you more fit to survive and breed.
Also usually among humanoids, the more magic-inclned folk tend to be very small. Only some humans are born sourcerors or become wizards, but every pixie, fairy and gnome is a natural caster. Perhaps, for humanoids, height interferes with magic. You could explain it as every humanoid having the same amount of magic energy, but the more mass you have, the less enchantment you have per kilogram of body weight. The effect of this is that along humanoid speciation, you would have a disruptive process... On one side, giants, orcs, humans etc. who rely less on magic and more on muscle. On the other hand, you have a plethora of creatures who rely on magic to survive. For those creatures, mutations that cause them to be taller are deleterious.
"Historia de gentibus septentrionalibus", a book by Olaus Magnus, a swedish writer and geographer, has been published in 1555: https://en.wikipedia.org/wiki/A_Description_of_the_Northern_Peoples .
The book contains a description of a race of pretty short people who live in Greenland. According to the book, the sky dome is located at such a low altitude over the ground in those northern lands, that only rather short people could stand and walk there at full height without hitting the sky with their heads: http://runeberg.org/olmagnus/0157.html