In fantasy, a relatively common trope in fantasy settings, particularly D&D, is underground societies that are more willing to endorse slavery than surface-dwelling societies.
We see it in the Drow, the Illithids, and the Aboleths (they're more underwater than underground, but they are prevalent in the underdark), but we don't get a reason for why these societies endorse slavery other than an explanation that basically boils down to "they're evil, that's why they've been driven underground".
I want to explore a similar trope but want a reason for why these underground societies endorse slavery that is tied more to the environment they occupy than the societies themselves. Slavery is wrong, and the societies that endorse it are obviously making a VERY BAD moral choice, but I want a reason why the underground caverns in my world have a higher concentration of such societies than the surface.
Basically, I'm asking; What kind of geographical conditions might make underground environments more 'palatable' for slave-owning societies? Note that the setting is a low-fantasy one where magic is rare, and the tech level is late medieval.