Upon disincarnation, consciousness remains, but it is no longer "attached" to the physical body. Consciousness is not made of matter, and therefore is not bound by "two bodies cannot occupy the same space at the same time". No matter means no mass, so no gravity.
So far so good, right?
For some reason, though, the disincarnate mind still retains most, or even all of the memories if had while attached to a body. And here is the thing about not falling through the floor: the living have a habit of not falling past a perceived ground. And habits are harder to break than atoms, so your ghosts stick around because they think that is what is natural, even for ghosts.
If a very pedantic, very physics-savvy person dies and realizes that they are now a ghost, and therefore not bound by gravity, while at the same time remembering that the orbital speed of our planet around the sun is around sixty-seven thousand miles per hour... They would shoot out into space at thrice the escape velocity for Earth's surface. This could be hilarious to some people, though not so much for the poor ghost. Most people don't know [expletive] about rocket science, though, so they don't have this problem.
Let's put a little more science into it.
Planets and stars "deform" spacetime around them. In very laysman terms, that's how gravity works (warning: boring to people who don't like quantum physics), when you think about it in relativity terms. We've all seen graphics like this:
Now let's throw more fantasy science into it.
Ghosts, as well as the living, exist in more than three dimensions - or four, if you count time (but I don't want bring relativity into the topic for now). And the three spatial dimentions are not the only ones distorted by gravity.
Your ghosts move in the 4th, 5th, 6th, up until the nth dimension if you want, and that's how they go through walls. These dimensions are distorted differently by gravity - so rather than accelerating towards the center of mass of the planet, you tend to stay put at the same point relative to that center of mass. That's why ghosts not only do not fall through the ground (and shoot out into space), they also need a willed effort to float up and down.
This could also explain how they disappear in one place and reappear in another. They are moving on other dimensions.
Last but not least, they interact with "physical" objects by reaching out to the boundary between whatever dimenions they are in and the three classical spatial ones. They can never go past the boundary (lack of a body and all), which is why they sometimes have a hard time pulling a respectable poltergeist.