Insular Dwarfism, exactly what produced Homo floresiensis
Reduced resources on islands drives smaller sizes in the majority of large animals who end up on them. This is exactly the sort of pressure that influenced the ~3ft 7 height of Homo floresiensis, and modern peoples of short stature like the Sentinelese/Andamanese. Interestingly, this also occurs in other places with limited accessible resources such as jungles (but not typically in cold places as cold temperature selects for increased body size to conserve heat).
There's currently no solid evidence that these smaller statures significantly influence intelligence in modern populations, and the current thinking is that Homo floresiensis was not significantly different in this respect to other paleolithic hominids. Homo floresiensis was also known to make comparable stone tools to other similar hominids, so was clearly dextrous.
Add in and environment where they typically operate in low-light situations, like the jungle floor or around dawn and dusk, and you have your improved senses sorted (although you may have some tradeoffs like reduced colour vision in exchange for better night vision).
A Potential Issue
There's an issue however, that if a species gets stuck on an island long enough to evolve insular dwarfism it gets very difficult to get off that island. The adaptation for small size, comparatively low population, and often a lack of predation means that species are ill-adapted to a more competitive mainland environment. Plus, physically getting off your island is difficult.
A Potential Solution
Luckily for you, hominids are some of the most resourceful species. There's some evidence that Homo foresiensis made their way to their island home via boats or rafts (given the distance they would have had to travel across the sea). If they retained this technology, and eventually found their way onto a polynesian-like archipelago (after evolving insular dwarfism), then they could potentially break out of their isolation. Especially if there's an Australia-like continent they could expand onto that's devoid of other hominids.
This scenario would eventually pressure them away from insular dwarfism, but you now have a spectrum of time in which to site your story where you can have potentially widespread diminutive hominids that haven't yet grown tall again.