An artificial, floating "moon" could work. Of course since it isn't in orbit, it's not a moon. Then again, we don't have a word for such an object, so calling it moon wouldn't be that far off...
It could have been constructed out of some kind of aerogel-like material in vacuum, so it would essentially be a solid, rigid foam filled with vacuum bubbles. It would need to be strong enough to withstand the pressure, non-permeating enough so gas does not leak in through the walls enough to matter, and with low enough density so it floats. If vacuum seems implausible, it could be filled with helium, and still be light enough to float on top of Methane/CO2/Nitrogen/Oxygen atmosphere.
Perhaps it was a construction project which got abandoned before completed Perhaps it was stripped of all technology when the ancients left it. Or perhaps it still has some ancient technology on/in it if that fits your story. If the point of your story isn't how the "moon" got there, then you can just not cover that. Or you can describe its structure, but let the reader to draw the conclusion that it must be artificial.
So it is just a solid, stable, floating object, big enough to withstand erosion for millions of years, which got there somehow. As the top side eroded and accumulated dust (eroded stuff from bottom side will just fall off), it would slowly get denser and slowly sink lower. It would also occasionally (every few millenia or whatever) turn upside down, as top side gets heavier in this process. Then the loose stuff on the top side would drop down, and it would rise in the atmosphere. So it would not be a good place for any life which couldn't anchor itself, as there's this cataclysm every few millenia...
It could also be hollow or have a vast cave network. Just note that it would not be a balloon then, the hollow inside would be filled with normal atmosphere and not provide lift, all the lift needs to be provided by the solid "aerogel" material.