In most fiction, the Overlord isn't all powerful or is in a weakened state. Even immortality has it's caveats: is it simply one cannot die from injury but still ages, cannot die from age but can die from injury, or cannot be killed but they have an otherwise normal body such that they can still be dismembered and scattered but still live?
Not being all powerful, the Overlord has to operate in secret. There's always a hero or some yin/yang force opposed to the Overlord that requires an agent or hidden cult to counter them.
Let's face it, an all powerful Overlord makes for a boring story: Chapter 1, the Overlord enslaved reality with a mere thought. The End. There's no conflict, and you can't have conflict if one side is completely superior in all regards. Even a complete underdog needs to be able to inflict a wound in defiance. In addition, in most games, the protagonist has to work his way through the ranks of trash mobs to get items/experience. It's a very short game if it starts you on the final boss.
Examples below:
Lord of the Rings: Sauron was weakened after losing the ring (shouldn't have been on the front line to begin with and lose it against Isildur, minions!). If he were to just storm out of Mordor, whoever he was pursuing with the ring would likely value over their own life over the charms of the ring and toss it in Mt. Doom to survive. He needed agents until he could find and corner them.
The Call of Cthulu: Immortal alien slumbering at the bottom of the sea in R'lyeh, demigods shouldn't need minions right? Time is relative to an immortal, after a couple billion years, that magic equinox that only lasts one night is a very very tiny window of time. There's also the issue of many of the elder gods not being used to Earth-rational physics and need to get a feel for walking in a universe with thermodynamics. Also to consider, being a giant elder god, humanity is kind of like ants in an anthill. Would you scoop through the entire anthill structure for those artifacts/annoying inspectors/would-be heroes or would you control ants to go about and do it for you?
Overlord (Light Novel): Ainz Ooal Gown may be all powerful in regard to his opponents, but he doesn't know that. For all he knows, the world he awakens from his slumber in could have a stronger hero on some other continent who hasn't hunted him down solely because the hero doesn't know he's there/awake. First thing Ainz does is gather minions for intelligence and labor.