A woman's birthing career, if she does little else, is about a pregnancy per year for 16 years, from the age of 16 to 32. Beyond 32, you run a risk of greater genetic anomalies, nearly always not good. A male is about the same, even though males are essentially fertile for life; the same thing happens with age; a greater risk of genetic anomalies.
But one man can impregnate almost any number of women, even one per day. You don't require a lot of men.
From there, presume there are no multiple births or child deaths; each women can produce 16 adults. The replacement value for the population of X is X/16 women breeding. That is about 6.25% of carefully chosen women.
Since the proportion of men required is far smaller, we can be more selective with the men. (Nature is too; for example in horse herds nearly all foals have the same father, that impregnates 95% of mares. A few rogue non-alphas manage to impregnate the other 5% of mares.)
So say, our breeder males impregnate a woman on average once per week, which is 52 times more often than women get pregnant, so the 6.25% of women require about 1 out of 832 men. So only about 6.37% of the population is breeding; one in 15.7 people.
For any decent sized population in-breeding should not be a problem; the odds of people having the same father in a population of 1 million is about 1200 to one (1m/832). (1200 is also the number of breeding males (1m/832), while the number of breeding females is 62,500).
It should be relatively easy to ensure every coupling pair has no common parent, grandparent, or great-grandparent. I believe that is the extent of in-breeding you need to worry about.
Edit: To preserve the idea of LuizPSR: If artificial insemination is an option, then the dictator can select his ideal couples, gather sperm and eggs to produce his new genetic specimens, and force non-selected women to carry those blastocysts to term and raise the children (to which they have contributed no genetic material). The "ideal" women would never get pregnant. Just harvest an egg per month from them.
This could drastically cut the number of women required. The only worry then is management to prevent inbreeding and ensure genetic diversity. Yes, it is a brutal form of slavery, but that is the fictional premise.
So, for example, if the dictator deems only 1% of women as breedable, then each of those women must produce 100x more babies than they would have if free. But that is possible, she would have produced 2 in her lifetime, she needs to provide 200 eggs, which she can do in 16 years, 8 months of monthly egg harvesting. Thus from the age of 16 to 32 +8 months. Those would be implanted into two hundred non-breeding females, as the eggs become available, and eventually all the non-breeding females will have two children.