The beginning of the end for the mafia was when drugs came involved in mafia business. The mafia made a pretty tidy racket doing some rather laughably mundane things prior to the hard drug trade. One such racket was stealing tanker trucks full of gas and using them to refill mafia owned gas stations. The gas stations would sell the gasoline without having to pay the taxes or purchase the gasoline. Rackets like this would bring in anywhere from \$15,000-\$20,000 a month. That's good money, but itsit's not "beat somebody to death with a baseball bat and ditch their corpse in the harbor" levels of income. People usually don't start committing heinous impossible to ignore crimes like torture followed by murder and mutilation over numbers like that.
What's more, when the big Dons started getting locked up by these new harsher sentences the guys who replaced them were not very smart. The real dons were replaced by street thugs in fancy suits. When you give a guy whose job qualifications are "breaks kneecaps real good" and "scares store owners into paying their protection payment without a fight" several million dollars and the reignsreins to a few hundred guys just like him things get... messy. Once the officers and bosses got locked up and the soldiers started running things is when the crazy over the top violence and endless cycle of back stabbings happened. You can't run an organized criminal enterprise that violently and get away with it for long. Thanks to mandatory sentencing decapitating their actually qualified leadership they started killing each other over the drug money they were making like feral dogs fighting over a steak, or got arrested for multi decade long sentences.