There are several factors that improve both your odds of humanity recovering at all and the time it takes to do so...
Time to prepare
If the world "sees" the catastrophe coming and can prepare for it, your odds go up. Maybe the preparations include things like secure bunkers designed with the tools, materials, and skills necessary to rebuild the world. Maybe the preparations are less centralized and more local, at the "stock up on canned goods" level. But this helps you.
Nature of the end event
Is the event man-made? Supernatural (zombies are so passe, but...)? Alien invasion? Meteor? Plague? Cthulhu rising? Chupacabra + Yetti + Bigfoot + Loch Ness revenge? Robot ninja zombie attacks? Not all apocalypses are created equal. If humanity must band together to overcome some external threat, the odds increase. If humanity tears itself apart in war, then the odds decrease. If the threat looks like us then you're generally going to find it harder to trust "outsiders." But if the threat is clearly not us, then we may overcome our natural fear of each other in favor of survival.
Speed of collapse
If the apocalypse happens slowly, then people have time to gather, to prepare, to work towards a future that isn't just death. This increases your odds. If you wake up tomorrow and almost everyone is gone because of some religion's (or religions') end of days event, then you will panic and seek out survivors. Odds go up. So you have this bell curve of survival. In the middle, with fast but not immediate collapse, where unrest and fear and panic can rule over common sense, this is where you find people setting up violent gangs and being generally unkind to each other. Wars, plagues, climate change, etc. fall in the worst zone for future friend-making.
Hope vs. fear
Generally, you need an environment where the survivors can find some hope for the future. If there's a flash-point and then the thing is done and over, then there's hope. People have a reason to live and work towards tomorrow. If the thing is persistent and will always be out there, striving to destroy the survivors, then fear will win and people won't be able to overcome that to work together.
mobility and communications
Survivors, after the end is done and everyone realizes they're alive and in the epilogue of the world, they have to be able to find each other. If all 5% are on the same continent, then, well, they have an easier time coming together and working for a better future. If they're so spread out that most survivors believe they are the only survivors, then your "epiloguers" must work much harder to come together. This is improved if the infrastructure required for internet access remains viable. See this story from Cory Doctorow that provides a good example of survivors coming together and using the internet for good.
If this is a purely random distribution, then you have big problems. Roughly 26% of the population will be below the age of 141. 8.5% of your survivors are over the age of 652. This means somewhere between 25-40% of your survivors are not really 100% self-sufficient based on age.
Meanwhile 46% of your survivors live in rural areas3. The other 54% live in cities. Cities full of rotting corpses (or zombies or...). Cities lacking in safe drinking water and renewable food supplies. These people are at high risk of death by starvation, malnutrition, and disease over the next year or three. Cities that won't have heat sources in higher latitudes or air conditioning in lower latitudes. Those summers and winters will be rough for them.
Also at a purely random distribution, you will likely lose all knowledge from at least some critical professions. For example, doctors represent roughly 0.1% of the world population4. By purely random chance, statistically, you've lost them all. Same for civil engineers and many other critical skill-sets needed to rebuild.
So you want to stack the deck. Make sure your survivors end up in rural environments where possible, as close as possible to ages 16 to 35, with skills, materials, resources, and such at their disposal, and with at least a few elders in the 30-45 year age range with specialty skills (medicine, engineering, etc.). You also want to pre-screen to whatever extent possible for diseases, genetic traits, or issues that would preclude them from having viable children or for diseases like Type 1 Diabetes that require highly specialized or difficult medical treatment.