We all tell ourselves it's awful, because it's less awful than accepting that we're missing out...
I have noticed that many people are attached to the idea that immortality would actually be kind of awful. Though these same people may, to varying degrees, dread death, deplore the depredations of aging, and wish for more time to do the things they want to, yet they insist to themselves (and anyone else who will listen) that living forever, in the flower of youth, wouldn't be that great. In fact, it would definitely be worse than growing old and dying. I mean, you wouldn't appreciate life, if you had more of it. Maybe you wouldn't be motivated, because you'd have too much time to actually buckle down and accomplish anything.
Pizza is fine every other week - but if you lived forever you'd definitely get sick of it. I swear.
Now, all of this is said in a world where it's fully plausible to believe that nobody lives forever. Imagine how much more emphatic we short-lived creatures would be about not regretting our mortality if there were other people living alongside us, who were youthful and vigorous for hundreds or thousands of years - or forever! In exactly the same way you might watch with longing and envy as someone else eats a perfectly ripe peach - before you declare to yourself it was probably mushy and flavorless - we might easily look on to the longer lives of our neighbors and insist their lives are not as sweet as ours.
Then add to that the further injury that these neighbors are not only immortal, but also able to make their perfect peach out of cardboard and a wave of the hand. They are able to call up mansions (a different mansion every day, if the whim takes them) out of air and dirt.
Surely, surely, such good fortune has its own kind of poison and disappointment, somehow buried deep inside. I'll bet they struggle with apathy. It's not me who is bitter and apathetic as I watch the magic I can never have, the eternity I will never enjoy. I'm not at all envious that I must content myself with my daily labor and ordinary meal.
And we would urgently try to convince not only our ordinary neighbors, but also our magically gifted neighbors, that the magically gifted are the ones truly missing out on something.