We have an example of this kind of divergence with Bonobos and Chimpanzees. They look almost identical, but have diverged enough to be considered different species. They are very similar but have different characteristics. This could be a basis of consideration when pondering how evolution would have created both species, even though we are talking about something pretty different.
So look at pressures that would create your desired characteristics.
Most of them could come from a single (and fantasy consistent) Biome. A Rain Forest.
Here is how it could work. The majority of the continent they develop on is covered in dense forests. Perhaps they came over a land bridge or something a long, long time ago back when they were an Australopithicus. They had not fully advanced into the naked pink Terminators Homo sapiens ended up being. Since they entered the vast forest, the pressures are there to maybe create an elf.
A big, lumbering ape isn't going to perform quite as well moving around in trees as well as a smaller ape. A heavy body cannot climb as high, and height is where relative safety is. This gives you a shorter stature and lower mass. A stronger grip is also advantageous for climbing.
Other characteristics could also be advantageous in a dense forest. The canopy is going to lower light levels quite a bit. That, over time, gives better night vision. Other animals with good camouflage means that excellent perception means you can eat and maybe avoid being eaten. Same with hearing. If you can't hear the snake coming up behind, you don't get to reproduce. Pointy ears that can move like a cats might result, but that could be a bit of a stretch.
Endurance could be sacrificed in favor of speed and agility. In a dense forest, you are probably better off as an ambush predator, as Gustavo's answer mentions.
So a dense rain forest gives you the pressures you need. Plus, they are elves. and elves go with trees like...well, you know
