You are thinking about something like Trantor. It is extremely unlikely something like this to happen. Reason is a simple energy balance.
You have to feed your population, and provide it with energy. For that you need fields: area where you produce food, open to sun. That is area where all kinds of animals can live. Most of such land is marginal, and will not be used for agriculture. And even more area will need to be devoted to producing carbon-neutral energy. Without carbon-neutral energy, your climate will collapse and productivity of your field will collapse, and your population will starve.
To increase population density, you will have to import food from dozens of other planets, and those planets have to be also habitable to be able to export food. What would Earth export in return? And where the energy to lift Earth's export will come from? And if humanity has access to such huge reserves of energy and many habitable planets, why not to move there?
Trantor was a center for huge fictional galactic empire based on Roman Empire, with many worlds devoted to farming, and with no robots to do the farming and other tedious work.
I think that your ability to supply carbon-neutral energy for your over-populated will run out sooner than area for farming and wild places like mountains.
Humans do not have to stagnate - they need to "get to the stars". Plenty of space in Space.
Some more thinking... I think OP ever-estimates plausibility of future overpopulation. Even today, person in technologically advanced society needs half year of education (with no income) for every year of work for income - elementary school to graduate degree. In the future, time invested in education to gain skills necessary for paid work will increase, and need for low-skilled work is steadily decreasing. Such pressure has same effect in all developed societies: decrease of female fecundity. USA is exception, not a rule: most other advanced societies have decreasing population, and not increasing.
r/K selection theory distinguishes between two population strategies:
- r strategy: produce as many offsprings as possible, and spend little resources supporting them
- K strategy: have few descendants, but spend lots of resources to make sure they succeed
I claim that K strategy is better suited for technological future of humanity. In the future, maybe even right to have a child will have to be earned (because r strategist cannot expect responsible taxpayers to support his or her r strategy, while limiting own strategy to K because of limited resources). Humanity will have to face limited resources, and adjust morality accordingly. Will it be forced sterilization after second child for a woman on welfare? Effective contraceptives in public drinking water? If every child has right to be loved and has access to opportunities, it forces K strategy on humanity.