Replace the humans with your offspring.
The only thing that can reliably bring about the extinction of members of our genus (Homo) is other members of our genus. The Neanderthals, for example - relics of their genome persist in ours, but as a species they are gone, supplanted by us. In a mere 100,000 years, modern humans have spread over the world and supplanted any members of our genus that were there before.
This is the plan for your immortals. They can play the long game. They will breed. They have a great advantage over Homo sapiens in that they are immortal. The parents will reproduce again in the next generation. This gives the equivalent of 2 extra offspring per generation. Additionally, the old ones are wily and wise, with accumulated resources. Almost all the immortals will be old, and they will handily outcompete the callow youth for the resources they need.
Your immortals cause the extinction of humanity by replacing them. Let us conservatively assume each generation, each immortal produces another immortal. A couple of immortals would produce 2 more. So in the first generation there are 4 - the 2 children and the parents. In the next generation there are 8, and so on. In 22 generations there will be 9 billion of them. With 20 year generation time that is 20* 22 = 440 years. Is 440 years so long for an immortal?
I can hear it now in the comments - oh, people will be freaked out by the folks that never die! They will die them because of that! Maybe so. Or not. Suppose in your tribe you have Moses. He is spectacularly failing to die, and having lots of kids. And it is good for the rest of you to have Moses in your tribe. Aside from being blessed by God and saving you all that one time, Moses is really smart, and smart * old = really rich, and smart * old * rich = powerful. And he is on your side! Other tribes wish they had a Moses. No way are you going to try to kill him. Plus he would probably know you were thinking about it before you did, and kill you first.
Could surviving humans hide out from their kin in the mountains? Sure - like gorillas do.