Whats the furthest two hominids could diverge from one another and still be able to produce viable offspring?

Question

In many fantasy stories humans are capable of inter-breeding with different fantasy races. There is a precedent for this happening with neanderthal and Denisovan DNA being found in people of Eurasian descent so we do know that humans inter-breed with other hominids. In my story I have many fantasy races such as Orcs, Goblins, elves, Giants, etc. So what I want to know is what is the furthest back two hominids could have shared a common ancestor and still be able to produce viable offspring(offspring doesn't have to be fertile it just needs to survive outside the womb and be somewhat healthy)?

NOTE: Magic does not exist in my story.

Answer 1

There's probably no way of knowing properly without consulting with a team of genetic scientists as the DNA-level intricacies of speciation and viable offspring, but my (extreeemely) rudimentary understanding is that procreation involves the 'zipping up' of two DNA halves from the mother and father's egg and sperm cells respectively. Problems, and eventually complete inability to concieve occur as the attempt to 'zip' these two sides of the DNA fail increasingly.

i.e. if you tried to patch-together two sides of a different outfit and have the zipper work, it would depend on just how different the zipper teeth are from one another - a small amount of difference would be fine, but the more drastic the difference between them the likeleyhood of the zip working in a way that is sustainable (i.e. you can keep zipping it up and down) lowers. Obviously DNA is much, much more complicated than this, but that is to some degree my understanding of it.

So for example, we share more than 50% of our DNA with a lion, but the remaining percent we don't have in common is drastic enough to critically fail in zipping up completely, whereas a lion and tiger are much more similar, and while the dna will zip, it will do so with such a number of problems the creature (a liger) will be sterile and have a plethora of other genetic issues.

However, if you look at the drastic differences between breeds of dog, you'd be forgiven for thinking they're all different species, but most of those differences truly are quite superficial, and are changes along lines that can neatly zip into the expected 'teeth' of each side of the helix (longer legs, snout length, fur colour, temperament tweaks)

To conclude: If then, you treat the various fantasy races as (to put it bluntly) 'different breeds of dog' in your story - to justify that really they are all pretty much the same base archetype with a few superficial differences accrued in a relatively short period of time (I'd say no more than 20-30,000 years, given we've domesticated dogs for around 10-15,000, and done so ususally with one single aim in mind with each new breed) your fantasy races could diverge quite drastically from one another to meet the classic archetypal appearences of orcs/dwarves/ogres and such, assuming the complex internal workings of their bodies are roughly doing the same stuff.

Which seems perfectly in line with most classic fantasy (most fantasy-races are capably omnivorous regardless of preference, most have the same sort of brain, with the only real differences being sizes of this sector or that (e.g. you could give elves a big prefrontal cortex to give them better foresight)).

Answer 2

A good example of which species can produce fertile off spring and which ones cannot are penguins. There are a couple of different penguin species and if you order them by average size of an adult then neighboring species can produce fertile offspring but species farther apart cannot.

Answer 3

As you mention, this is something we have real-world evidence for

There's some good real-world examples in our own evolutionary history. There are two known interbreeding 'events' between homo sapiens and other related hominins, and I've heard of evidence of at least one more interbreeding event with an unknown 'ghost population'.

Homo neanderthalensis

There's genetic evidence that sapiens interbred with neanderthals in Europe, enough to contribute nearly 1.5-2.1% of the DNA of all humans outside of Africa. Our common ancestor was Homo heidelbergensis, which we diverged from about 300,000 years ago and neanderthals did about 240,000 years ago. The earliest suspected interbreeding period was around 110,000 years ago, and the latest around 47,000 years ago. So we have a potential window of 190,000 years of divergence, after which interbreeding was evidently still possible.

However, there has been no evidence of neanderthal mitochondrial DNA found in sapiens population. This suggests that while male-neanderthal/female-sapiens parings produced fertile offspring, the offspring of female-neanderthal/male-sapiens parings were infertile. This could also be suggestive of a very low rate of interbreeding between the two (sub)species, but might indicate that sapiens and neanderthals were nearing the point at which they couldn't interbreed.

Homo denisova

There is also genetic evidence of an interbreeding event between sapiens and denisovans in Asia, which contributes between 3-6% of of the DNA of Melanesians and aboriginal Australians (and a lower percentage in Southeast Asians, including Tibetans). The ancestry of denisovans is less clear than sapiens or neanderthals due to the limited number of finds, but best guesses suggest they're more closely related to neanderthals and split from them around 160,000 years ago (possibly interbreeding with another, unknown archaic human population).

Current estimates put the interbreeding event between sapiens and denisovans at around 44000-54000 years ago. If denisovans are derived from neanderthals, that would put a window of around 246,000 years, after which interbreeding was still possible.

Similarly to neanderthals, there is evidence that sapiens-denisovan pairings weren't always fertile. There are large parts of modern human genomes that are devoid of any denisovan DNA, the location of which could be explained by male hybrids being infertile. This would also suggest that humans and denisovans were approaching the point at which they could no longer interbreed. However, due to the scarcity of denisovan finds there are a lot of questions still unanswered.

Morphological differences

To find out the level of morphological differences you're likely to be able to support via natural genetic divergence you can look at the comparisons between sapiens, neanderthals and denisovans. As a starter for ten the latter two were more robust than sapiens with more pronounced brow ridges, and denisovans had different dentition.

Another approach: artificial breeding

If you're looking for greater morphological differences, but retaining the ability to interbreed then you could look at dogs. It would take some creative circumstances, but perhaps all your various breeds of hominids were selectively bred by a historical (and now absent) population. This could conceivably allow for some very distinctive physical differences with minimal genetic divergence.

A cursory google indicates that there might be some concerns around the ability of some very isolated dog breeds to produce fertile offspring, but the general consensus is that even dachshunds and great danes can produce fertile offspring. From here, we're talking about purely mechanical problems with the ability to produce offspring (I wouldn't want to see a non-giant try to give birth to a giant-hybrid).