The problem is with mobility. In medieval-esque setting, about 80% or more people was working in agriculture, lived in villages or small towns, and generally had no business in traveling further than nearest merchant buying grains i bulk. For more details see Agriculture in the Middle Ages on Wikipedia.
As long as your mages have nothing to do with farming, processing farmed grains, or trading agriculture products, or collecting taxes from villages, literally each and every other person in your world (that is somewhere between 15 and 20%) could be a mage or witch and 80% of the people would never encounter one.
I guess that your story will not be about farming communities, but about the rest 20% of society. Then, it is getting trickier. Craftsmen and merchants generally met everyone in their vicinity - most towns couldn't afford to have many shoemakers, innkeepers etc and it was merchants' job to know and be known.
To limit chances of mage encounters, you need to not have one in towns smaller than, say 1000~5000 people. Magic user would be a celebrity in such a "small" city, so we can expect people would actively seek to encounter him and everyone would know where he lives.
Remembering that in early middle ages London had only 10`000 residents, we can assume that to meet your criteria interpreted with exclusion of farmers you would need to limit mages to capital cities, and maybe "cathedral" cities of your countries. How many would be there in such city is almost irrelevant, because again they probably would be local celebrities and people would actively try to encounter them.