First of all, we have to recognize that before Newton's laws of motion and law of universal gravitation, and before the advent of telescopic astronomy, the only two criteria which an astronomical model had to satisfy were elegance (how simple it was) and accuracy (how well it could predict the positions of the planets). There was nothing else to judge.
To understand why the ancients didn't fall for a heliocentric model, we must understand what Ptolemy’s, Copernicus’s and Kepler’s model were.
Ptolemy's model works like this:
Earth is a sphere, fixed and imobile.
Everything else rotates around the center of the Earth in a uniform circular motion, completing a rotation in 23 hours, 56 minutes, and 4 seconds (a sidereal day); this is called the first or diurnal motion.
For the fixed stars, that’s all there is.
Each of the seven classical planets (the Moon, Mercury, Venus, the Sun, Mars, Jupiter, and Saturn) executes two supplementary circular motions:
A circular motion along a path called the deferent, the center of which is not the center of the Earth, but a point called the eccentric; this second motion is uniform with respect to a point called the equant, which is placed on the line from the center of the Earth to the eccentric, at a distance from the eccentric equal to the distance between the eccentric and the center of the earth — that is, the eccentric is exactly midway between the equant and the center of the Earth. This motion describes the movement of the center of the third motion. The duration of one complete revolution of the second motion is what we call orbital period.
A uniform and circular third motion on a path called the epicycle around the point executing the second motion.
Copernicus’s model is almost the same, just considering the Sun fixed. Earth rotates around its axis (replacing Ptolemy’s first motion) and moves on an epicycle which moves on a deferent which revolves around an eccentric with uniform angular speed with respect to an equant (thus mimicking Ptolemy’s second and third motions) etc. There is nothing to choose between Ptolemy’s and Copernicus’s models, they are almost equivalent from a kinematic point of view; they describe very similar motions, differing only in the frame of reference.
Kepler’s model is radically different:
The sun is a fixed and immobile sphere with respect to the fixed stars.
Earth, Mercury, Venus, Mars, Jupiter, and Saturn move along elliptic paths, with the Sun in one of the foci; their motion is such that the line from the Sun to the planet sweeps equal areas in equal times (areal uniformity).
The cube semi-major axis of each ellipse is proportional to the square of the orbital period.
The Moon executes a similar motion around the Earth.
The Earth rotates around its axis, completing one rotation in a sidereal day (23 hours, 56 minutes, and 4 seconds).
Kepler’s model is kinematically different from Ptolemy’s or Copernic’s models, which are equivalent. That is, when computing the future position of the Sun, the Moon, Mercury, Venus, Mars, Jupiter, or Saturn at a future date, Ptolemy’s and Copernicus’s models give very similar results, while Kepler’s gives a different result.
Johannes Kepler was stimulated to develop a new model because he had access to the fantastically accurate observations made by Tycho Brache at his fantastically expensive Uraniborg observatory. These observations did not match the predictions of the Ptolemaic model. They matched the predictions of the Keplerian model. Game over.
As history is prone to irony, within Kepler’s lifetime the arguments in favor of his model accumulated: Galileo Galilei discovered the Galilean satellites of Jupiter, the phases of Venus (which were simply impossible to explain in a Ptolemaic framework, but were explainable in the kinematically equivalent Copernican model), and the rotation of the Sun around its axis. Nevertheless, the only decisive advantages the Keplerian model had were its simplicity (only eight motions instead of 22) and its better accuracy.
Then came Newton, and the Keplerian model was shown to be a direct consequence of three simple laws of motion and one simple law of universal gravitation.
So, how could a heliocentric model be favored in earlier times?
First of all, the rotation of the Earth could have been proven using an experiment similar to Foucault’s pendulum. There is nothing in Léon Foucault’s 1851 experiment which could not have been done in 1581 ,or 1081, or 581 or even 81 CE. They just didn’t think of it — and they did not study pendulums very carefully before Galileo. They could have, but they didn’t.
Then there was nothing revolutionary in Tycho Brache’s Uraniborg. It was the most advanced naked-eye observatory ever built simply because it was the biggest and by far the most expensive: to build it, king Frederick III of Denmark allocated 1% of the state’s budget per year for five years. No astronomer before or after Tycho Brache ever had such budget.
Kepler was very good at numerical calculations and very dedicated. When Tycho asked him to recalculate the orbit of Mars in accordance with the new precise observations, Kepler first set to compute it using Ptolemy's model. After many months of calculation he achieved an accuracy of 2 to 8 arc-minutes, but he wasn't satisfied; he then began trying to replace circular motions with various ovoid shapes; at the 40th attempt he tried an ellipse and his law of uniform areal motion: and it worked to better than 1 arc-minute. In the antiquity, only Archimedes was as good and dedicated at numerical calculations, and Archimedes was not interested in astronomy.
