I'm trying to write a mermaid civilization, although I have one major roadblock when developing a criminal justice system. My mermaids are air breathers (like cetaceans) so I'm not sure how I would have a system of detainment without drowning them to death. How exactly could I get around this hurdle?
You don't need prisons
"I'm not sure how I would have a system of detainment without drowning them to death." Well, don't have a system of detainment then. Public drownings are the social event in town. There are snacks, you get to bond with your neighbors and you get to make bets on which prisoner will last the longest.
Imprisonment is expensive. This has always been the norm in human history. Have a system of either corporal punishment, mutilition, banishment or death sentences. Slavery could be combined with branding criminals.
For detainment you could just use a rope around the neck and their hands and guards. Alternatively, you could bind groups of prisoners together so that they can't escape unless they reach a marvelous level of coordination.
I present to you, the most feared and inescapable prison of the merfolk world:
There is an island in the middle of the ocean. And in the middle of this island is a massive saltwater lake, connected to the sea by a singular, underground, underwater tunnel the merfolk have dug over time. There's nothing keeping the merfolk imprisoned therein from accessing the surface. In fact, there aren't any guards around the outer rim of the prison at all. Because there's only one way in, and one way out. And that's where the vast majority of the guards are.
Anyone who tried to escape any other way would have to be doing it over the surface, dragging their legless bodies across hot, inhospitable land crawling with dangerous land predators. Some have tried escaping that way. The ones that didn't die in the attempt quite literally came crawling back when they realized how out of their depth they were.
Just build the jail at the surface, so the cells are partially submerged but still have access to air.
I imagine this would be required for all buildings in which merpeople are expected to spend long periods of time, so building a jail should not be a particularly special project (or at least, no more so than it is for us land dwellers).
If you don't want cages to breach the surface, just chain the mermaids to the ocean floor. That way, they can come up for air as they please. Of course, keep guards handy to fight off whatever sea monsters might come by.
Alternatively, install a long snorkel from the cage to the surface, with a simple valve at the bottom to stop water from filling it. You can disguise it as a floating seaweed blob or Portuguese Man of War jellyfish, if you're paranoid about surface-dwellers noticing the tube.
Any of the above as suits plus Salmon Farms
A frame supported by buoys or floats and/or ropes & chains anchored to the sea floor. Then you cover this with layers of stiff netting. The walls rise a few meters vertically above the surface and curve inwards towards the top.
Add some trained dolphin 'guard dogs' patrolling the perimeter.
EDIT: even easier just deploy something similar across the entrance to any small bay or estuary that suits.
Merfolk jails need not chain nor ball
Merfolk don't have prison in the traditional sense. Instead of capturing and incarcerating each other in physical shape, they capture their criminals and then partake in a sort of partial ritual canibalism, consuming the arm of the criminal by the community: each person in the courtroom takes a bite. Gifted with a strong healing, that arm will regrow eventually, but until then, the criminal's essence is entwined with the society and they can not do anything against society. In fact, because those other members of society have absorbed part of the criminal merfolk's essence into theirs, they know where the criminal is at any time - there is no way to flee, and a whole bunch of people will know if they violate their terms - and could come back to punish them even further.
A fjord is a long, narrow bay that was carved by a glacier and typically has very steep sides. Build a fortified barricade/gate where it opens to the sea, and you have a nice isolated place to store your criminals. Even if they could travel awkwardly over land, there's not really an overland escape route that doesn't involve climbing a mountain.
Fjords also tend to be found clustered together. You can convert several nearby fjords into a prison complex, with prisoners segregated based on the severity of their offense and their level of danger to others.
Convicted mermaids are sentenced to live with a dolphin pod
Mermaids can communicate with marine life (at least with the ones that have certain brain development like dolphins, and possibly all other fish and marine mammals). As dolphins are the most intelligent marine animals, merfolk have closer relationships with the dolphins. Merfolk also have a system that they can assign convicted mermaids to dolphin pods. Dolphins will keep the convicted mermaids at bay, away from the merfolk society. Dolphins are faster and they are stronger in numbers so the convicted mermaid has no chance to try something funny, but he can freely swim with them and go to the surface to breathe.
Mermaids are not cruel like other civilizations and they have respect to all marine life including themselves.
Merfolk sometimes jail mermen in warm volcanic lakes
Don't worry about the females. Mermaids are always lovely, sympathetic, innocent, and honest. As soon as a mermaid needs to be sent to jail, a handsome merman (or land human friend) will come to her defense. Mermaids are not expected to do anything wrong.
Mermen is another story. Mermen don't hurt anyone, but they are sometimes known to exceed the speed limit, and when they don't pay their fines, they'll be sent to jail.
The merfolk jail: a warm volcano lake
For reasons of containment, the merfolk jail is one of the many warm volcano lakes on your planet. When you put a merperson in a volcanic lake, this warm water renders it in a state of utter lazyness. They won't be able to leave the water. Volcanic lakes are a trap for merfolk, they are so agreeable the prisoner will not escape.
Take a river, any river, that has only one route connecting it to the sea(assuming the mermaids live in the sea). Build an obstruction like a grate that mermaids can't get through somewhere along the river, spanning the whole width and depth. Just put a guard or two near the gate in the grate, and you'll have a prison. The mermaids will have a lot of areas to swim to, going upriver, but as long as the river doesn't branch towards the sea or other rivers or ends in a dead-end(like a waterfall), they won't be able to escape.
The beauty of this is that it is a very low effort, low maintenance prison. The mermaids are responsible for farming their own food in the river. Fish can swim in and out. And the only cost is a bit of upfront work building the grates and a handful of guards near the entrance. In fact, making them responsible for their own food will keep them busy and build up a hierarchy among the prisoners, making them have to deal with that instead of escaping.
Need more space? Take a river that has a lake somewhere upstream. More space for prisoners, yet the guards will still only need to guard one exit to keep the prisoners in.
For a bonus, a large group of guards go inside the prison once in a while to do an inspection, count the living prisoners, and detect escape attempts.
This jail should be filled with water everywhere as mermaids cannot walk because their lower bodies are fish like and therefore a very advanced waste management system would be needed to remove all the poop and pee they do. The other everyday things could remain the same.
There are several options for merfolk to create jails/jail-like structures. The simplest would simply be using a vertical pillory, but facing up and floating at the surface, one could also make them sideways however it would be more complicated.
~ = water level
= = wood board
o = small hole
0 = large hole
Another option would simply to build a partially floating cage on the surface, like this:
========= | Air | ~F|~~~~~~~|F~ |prison | =========
And a third option would to tie them in a shallow area so they can reach the surface.
a Brine Pool Labor Colony
Brine pools are toxic to marine animals due to their high salinity and anoxic properties, which can ultimately lead to toxic shock and possibly death.
Documentary excerpt of an eel suffering from toxic shock after diving into a brine pool.
Levels of 'jail'
Any society with valuables will have a storeroom that can be protected to keep people out, it's a small adaptation to keep a person inside. Hence the local 'jail' will evolve out of whatever structures the mermaids use to store and protect their valuables from theft, probably underground pits surrounded by guards. What prevents escape is not the bars on the windows but the armed soldiers who already guard this location.
The socially-connected will get house arrest and have their freedom-of-movement restricted, and other privileges taken away (depending on how privileged to begin with). They must check-in with a probation officer/moral authority. This assumes the perpetrator is perceived as worthy of rehabilitation.
For the poor shackles are a low-cost solution. A pre-industrial mermaid society would have rope and nets. They could create the mermaid equivalent of a ball and chain.
For the worst offenders, life-sentences, and unrepentant recidivists, they could wall-in a cell or build a tower designed to be difficult to escape. An authoritarian could (will) create notoriously remote prisons for political prisoners and martyrs. Rehabilitation is not considered; the goal is to break the prisoner, and propaganda.
Those whom society deems undesirable are sent to work at a brine pool labor colony. Death tolls are high. Few come back. Worldbuild a need for the brine pool (some industry or mineral resource), and your mermaids will discover a pretext why brine pool labor camps are a necessary part of the justice system.