My fantasy world has a fairly rigid system of dealing with the souls of the dead and ferrying them to the afterlife. However, this system is in peril and in danger of collapsing. This would mean that the souls of the dead would no longer be dealt with, and left to roam the earth freely. This should obviously be disastrous, but why?
The common two downsides in classical literature are ghosts gaining the strength to wreak havoc in the real world, or the ghosts emitting EMF (or equivalent) waves that disturb technology.
However, in my world, the ghosts directly interacting with the real world isn't a thing save through mediums or others with abilities. And the number of 'sensitive people' isn't all that high. The ghosts can talk to these sensitive people, but not to anyone else. No possessions, apparitions, interacting with material objects, etc. Meaning souls directly messing with the living isn't really an issue.
Also my world is low-fantasy with an Edwardian technology level (mid 1910's) vaguely British, so save the odd lightbulb flicker and jumbled telegraph message there's not a lot of technology EMF can influence on a large scale.
In case it is relevant for answering: The system in my world is that there are reapers and carriages who deal with the dead. If someone dies, a carriage comes to collect their soul and transports it through a portal to the afterlife, which then means it is no longer part of the current world. The reapers are similar to the conventional ones, save that they don't do the killing. They serve as the first point of contact to deal with the emotional state of the dead individual and lead them to their designated carriage. The secondary function of the reapers is to care for and coordinate the beasts that draw the carriages. The event that will happen is that something happens to the population of carriage drawing beasts, meaning there isn't anything available to transport the souls. The dead souls would still be soothed by the reapers, but then just left to their own devices. The dead can't use the portals to the afterlife by themselves or with help of the reapers due to the effort required.
Is there any other downside in conventional belief or classic literature that would make sense as a downside to this, beyond "those ghosts would be sad"? Preferably something fairly immediate, not something that would maybe be an issue in a few decades.
TL;DR What happens if you simply don't deal with ghosts who don't go rogue or submit EMF?