There's two factors that you need to worry about: chemical weathering and mechanical wearing.
For chemical weathering purposes, perhaps the best rock to last through the centuries would be slate.
Remnants of rock after chemical weathering -> silt
Silt compressed into rock -> Shale
Shale metamorphosed -> Slate
Since slate is compose of only mineral grains that survived chemical weathering, it is of necessity highly resistant to chemical weathering. Unfortunately, it really isn't resistant to mechanical weathering. So if not subjected to mechanical weathering, your best bet is slate. It would likely last several centuries for rain to erode the house.
To resist mechanical weathering you want a rock that is very hard and tough. Granite (and related rocks like andesite & rhyolite) are composed of multiple different minerals. In combination they act like a composite material with the different materials providing some of their better qualities to the overall structure.
Unfortunately, several of the minerals in granite are highly susceptible to chemical attacks. Stroll through a cemetery and look at the granite headstones. After a few decades the chemical weathering begins to make reading those headstones quite difficult.
But if you took a hunk of slate and bashed it into a hunk of granite, the granite is going to win - every time.
For water to erode the structure will likely take 1-2 centuries.
Since you stated this is a post-human world, I'll make the assumption that weathering is the primary factor for your building - which means Slate is the ultimate winner.