Mutated plants and animals and alteration of weather patterns suggest a nuclear winter as a plausible explanation.
In terms of the effect on humans, the main thing you need is a way of disrupting parent-child transmission of knowledge. In the case of technological knowledge, the easiest way to disrupt this is to simply have several generations for whom such knowledge is entirely irrelevant and they're unable to practice it - eg even if your great-grandmother told stories about driving a car, if you've never even seen a car that's functional enough to drive, you won't have a clue how to drive one yourself.
For more social advances, such as the scientific method, germ theory, etc, it'd be harder to explain that knowledge disappearing. Rather than thinking ancient technologies are magical artifacts to worship, it might be more plausible if people continue knowing that their ancestors build those technologies based on scientific principles, despite having forgotten the exact methods used.
However, one way to have such knowledge disappear would be to have the people who could have transmitted that knowledge actively reject it. There are people living in modern society who reject modern scientific understandings of the world, such as anti-vaxxers, Creationists, flat-earthers, etc. If for whatever reason these people were isolated from the people who embrace scientific understanding for a couple generations, their descendants wouldn't learn about those theories, and would grow up with a much more medieval understanding of the world.
You say there are technologically advanced people in bunkers? Maybe the people who actively saw the benefit in staying in a bunker were generally readily able to find a bunker to stay in, but a significant contingent of people formed conspiracy theories about how the impending disaster was a hoax and the bunkers were a trap to bring us all under more direct governmental control, and so the people who stayed on the surface were disproportionately people inclined to reject well-proven scientific facts. Given the overlap between the above-listed groups and climate change deniers in modern day, that seems like a very plausible scenario.