Many things to discuss here.
1. Human Nature
First, the reasons characters act the way they do is because it's literally human nature. We have evolved to be terrified of death, it's what kept our ancestors alive long enough to make a new generation. Whether or not one considers it logical to be afraid of death we will fear it, because that's how our instincts and our culture and everything around us tell us to feel. Fears are not always logical and some are very ingrained in our psyche.
2. What we leave behind
Fear of death can also be fear of what is left behind. To give an analogy people may be afraid of a move to another state even though they expect to live a decent life after the move. They're afraid because they have to leave their friends and family and everything they have known behind. They are moving on to something different and regret what they have to give up. The same can apply to death. You won't see you family again, you can never play with your son or go out for a night on the town with your best friends. Everything you knew and loved before you died will never be an option again.
3. Unfinished business
Closely related to the above, there is business that you may have wished to complete that you can't. Perhaps you defined yourself by your legacy you intended to leave behind and that legacy isn't complete. That startup business you were so proud of will collapse without you at the helm, or you were just about to star in a play you dreamt of being the lead in all your life and now that will never happen. Or a more likely, and worse, option, perhaps your young children will have no parent left to care for them once you die, they will grow up in an orphanage and never remember your face or how much you cared for them. Leaving loved ones that depend on you without your support is something anyone can rightfully be afraid of.
4. You still don't know what awaits you.
Most of these worlds depict some sort of life after death, but many aren't specific on just what it entails. Have you ever read Greek mythology? They had a life after death, but even the most heroic (which is as close as Greeks got to 'good') were looking at a bleak and miserable 'life' after they died, to the point that when Odysseus visits tartarus in 'The Odyssey' friends of his who had died tell him flat out that death is horrible and he should do everything in his power to live as long as possible and enjoy life to avoid this fate.
Unless your certain you know what the afterlife entails there is still plenty to fear. It's still a great unknown, there is no way to be certain if the world beyond is something to be enjoyed or hated. There is reason to be afraid of that unknown.
5. Are you certain you're headed up?
Lets say the christian version of Heaven has been confirmed to exist, doesn't that also imply a Hell? Who goes to Heaven and who goes to Hell?
Even within Christianity we can't seem to decide that. Some say that anyone that believes in god goes to heaven, but do you really believe, have you never had doubts? Even the most pious of men surely have, but how severe can your doubts be before they exclude you from Heaven? What of the religions that have strict rules you must follow to earn heaven, have you followed all of them? Surely every man in the world has broken one or two of those rules, that's human nature. How many are you allowed to bend before you no longer qualify for Heaven? How many good deeds were you required to do, and did you do enough?
In short there is no promise that you will be the one in Heaven. Obviously anyone facing Hell would have every reason to be terrified of death, but since it's hard to tell what standards you must live up to, or if you met them, everyone, no matter how pious or altruistic, should be aware of the possibility of Hell.
Then there are all the atheists, agnostics, and other religions out there. Only one religion can be right surly, that means every other religion is doomed to hell by default! Even in a world with a confirmed afterlife some may be atheists, some may disagree with the values from the bible (or other religious text) or feel God doesn't deserve their support if he refuses to intervene to help people, all those would presumably be headed to Hell.
6. It's a self selecting demographic
If you think about it those that aren't afraid of death, and are confident that they will be headed to heaven, have no reason to stick around this world. Death is not hard to achieve, if you're certain what lies beyond will be better then your find a way to make that happen. The only ones alive now would be the ones that have sufficient fears about death to have survived.