I can imagine a building about 100 feet (30.48 meters) wide and 1,000 feet (304.8 meters) high (almost 100 floors) and 1,000 to 10,000 feet (304.8 to 3,048 meters) long. It has widows in the rooms on both sides and broad corridors down the long middle of the building. And the broad corridors might have relatively steep stairs for the athletic and relatively gentle ramps for others. Some of the ramps might be used by pack animals to bring supplies up and some of the ramps could be broad enough to drive carts with supplies up.
I thinks that horses, etc. would usually fear steps and would have good chances of stumbling on steps and falling. The Scarlet Empress (1934) has horses charging up stairs, but that is a scene in a movie. But gentle ramps should be okay for horses.
The Abbasid Caliph al-Muktafi (reigned 902-908) built the Palace of the Taj and many other buildings at Baghdad.
Ali Muktafi also constructed halls of assembly and divers cupolas in the immediate neighborhood of the Taj; one especially was known as the Cupola of the Ass (Kubbat-al-Himar), this being a tower ascended by a spiral stair, of such an easy gradient that the Caliph could ride to the summit on a donkey trained to an ambling gait. Thus without fatigue he could enjoy the view over the surrounding country, for the height of this tower is described as very great, and in plan it was semicircular.
The Rundetaarn in Copenhagen built 1635 to 1642 has a helical ramp to the top.
This design was chosen to allow a horse and carriage to reach the library, moving books in and out of the library as well as transporting heavy and sensitive instruments to the observatory.
People have ridden horses, bicycles, and motor vehicles up and/or down the ramp. It is said that Empress Catherine I of Russia rode a carriage up the ramp in 1716.
Legend claimed that you could drive a carriage up to the top of the tower of Fonthill, but the tower was actually too narrow for a sufficiently wide spiral ramp. But a series of long straight gently sloping ramps would be fine for riding horses and driving carriages and carts hundreds or thousands of feet in the air.
The first rail roads were for hand or horse pulled carts in mines, and thus indoors. And I have read that King Louis Philippe of France (reigned 1830-1848) had a miniature steam train which he used to ride in the Grand Gallery of the Louvre - inside.
So a tall building could have railroads that hauled loads of supplies up and down long straight or wide spiral ramps. Or steam engines could power winches that pulled carts up ramps and lowered them down.
Possibly the stairs and ramps would not be in the central spine of the building but at intervals at right angles to the building. That would avoid problems with steam engines making indoors air pollution.
A system of windmills could pump water tens, hundreds or thousands of feet to the top of the building when the wind was blowing and store it in reservoirs near the top for later use.
Most of the length of ancient Roman aqueducts consisted of underground water channels. Sometimes when an aqueduct had to cross a valley it would simply go down the side of the valley and back up the other side, relying on the water pressure to drive the water back up the other side of the valley.
The Romans only put their aqueducts on top of tall arches when the ground was sloping too steeply for a steady flow of water.
And sometimes the Romans built tall arches to carry aqueducts across valleys. The tallest Roman aqueducts and viaducts (bridges) include 19 with maximum heights of 28 meters (91.8635 feet) or more, 13 with maximum heights of 100 feet (30.48 meters) or more, And 5 with maximum heights of 40 meters (131.234 feet) or more - up to 66 meters (216.535 feet).
Here is a link to an image of the Pont du gard aqueduct and bridge near Nimes, France.
It has a maximum height of 48.77 meters (160.00656 feet).
Now picture 2, 3, 4, or 5 parallel copies of the Pont du gard across a valley, close enough together to lay floor beams between them. Fill in the arches with masonry walls and glass windows and put a roof on the top and you have a building that is up to 48.77 meters or 160 feet tall. There can be access roads that gradually rise up the sides of the valley to reach doors at each level of the building, so that people who live or work on one floor of the building have little need to use the stairs and ramps inside the building to go up or down. And that building across a valley can be built at one end of an aqueduct that carries water from higher elevation down to it - the Pont du gard that inspired it was part of an aqueduct.
One possible purpose of such a building would be to be a fortress across a valley that could be used as an invasion route.
So with advanced enough building techniques, and sufficient reasons for building a building of a great height, and sufficiently intelligent methods of getting people and materials and water up to the top and/or down to the bottom, a science fictional or fantasy building could be built several times 100 feet (30.48 meters) or several times 1,000 feet (304.8 meters) tall without using anything resembling modern elevators.