Considering that humans evolved from "[...] a tree-climbing, furry-tailed insect eater that weighed between 6 and 245 grams" (Nature News), it is perhaps conceivable that small humans could evolve. After all we have mice, bats, flying squirrels and all sorts of other very tiny mammals.
The smallest of those are the Etruscan shrew at between 1.2 - 2.7 grams (1.8 grams on average) and the slightly more massive Kitti's hog-nosed bat at about 2 grams.
Interestingly these both evolved, from the above common ancestor ( which according to Nature has a probabilistic description, but no actual fossil record, and no suggested name in the article ), to be smaller than the smallest projected size of the common ancestor. Overall enough time has passed to see all of the variations that we see in class Mammalia in the world today, from shrews to humans to Blue whales.
But this does not answer the question of specific physical and mental characteristics that are ( as far as we all agree ) distinctly human. First let us go down the list of traits related to mental capacity since this is a large factor in determining the remaining resources to support metabolism beyond cognition. These cognitive traits are what traditionally makes us uniquely human, but have been shown to be explicitly not unique to humans.
- Complex langauge? - dolphins
Tool use - too long to list
Art - birds, primates, elephants
So it appears there is room for some creativity and tool use at the bottom of the mass spectrum, but it looks like language will be the hardest to explain at that level. However we are even beginning to question that based on how it seems birds are able to imitate human vocalizations, and that is based on neuron density. In other words, by packing more neurons into a smaller volume, the ( choose your example ) of bird that imitates human sounds, including words and phrases, is approaching the brain to body mass ratio of primates. They have smaller skull cavities, but they are lighter and they pack more neurons into those spaces, which means their bodies are doing a similar kind of trade off to primates as far as cognition vs physical movement. Our brains consume about 30% of our daily calories and it would seem these birds are nearer to that range than previously thought, but I would caution that much more investigation in this area is necessary for any more definitive results.
The smallest known primate is the Madame Berthe's mouse lemur with a body length of 9.2cm (3.6 in) and a mass of 30g. This fits into the size range of a Borrower and is the same class of animal, however these creatures are very different anatomically than humans. The things we are perhaps most interested in are the hands, primarily opposable thumbs, since tool use, tool making and so forth are required for a creature like a Borrower, and perhaps the posture and gait of the animal, since we want them to stand up and walk around like people.
Other features like hair, curved nails, etcetera, I think we can assume, are merely cosmetic. Maybe such creatures would opt for hair removal but it would perhaps depend on their history, culture, and knowledge of medicine and disease. For example, only a couple of centuries ago many people shaved their heads and wore wigs to help prevent head lice.
One fact about hair is that smaller things, i.e. smaller volume to surface area ratio means higher heat loss. Many smaller animals may need their hair or may have completely different ways to cope with heat loss. The naked mole-rat is a mammal which is effectively cold blooded as it does not regulate its internal temperature and so remains close to ambient temperature. Most commonly, in addition to a thermal coat, such as hair, small creatures must have a higher metabolism, which generates more heat, in order to maintain a relatively constant internal temperature. However, we can assume that Borrowers would wear clothing like us, and would perhaps consequently have/need less hair, like us.
As for opposable thumbs, we are the only species that we know of which has this trait. However there are other very dexterous appendages that can accomplish some, but not all of what a hand can do, like tentacles. Interestingly, octopods also are known to use tools. Other posters may respond on the mechanical feasibility of opposable thumbs for a small, Borrower-sized species, but I do not see any physical reason why such a structure is not possible or would not evolve given all of the advantages related to tool making and manipulation that we are aware of in the human species.
Upright posture and walking gait is another question. We are familiar with the prairie dog, which stands upright, but does not have a natural upright gait, as do humans, but rather a posture and behavior that seems to be associated with an evolutionary advantage of being able to see farther, to spot predators, while closer to the ground. For smaller species it would seem that undergrowth and low ground cover flora in most environments would negate any such gains and preclude such an advantage.
Furthermore, there is the physics involved in a very short standing posture. Consider this from the point of view of trying to balance a broomstick on your hand, verses trying to balance a pencil. The broom stick is longer and requires less attentive movement to adjust and keep it balanced, while the pencil is short, has a lower center of gravity and will naturally topple over much faster than the broomstick. This means that an upright posture for a very short species would require quicker and finer adjustments for the creature to remain upright. However these effects can be offset by the size of the base of the posture, by making the feet larger or foot and toes more spread out.
Mice can stand upright and beg for food or sniff the air, but they are forming a tripod structure, with large haunches, a wide stance and they are leaning back against or balancing with their tails.