I love this question! There's a few ways you can handle this.
In real life, most religions persist despite the fact that there isn't too much evidence of a divine being in recent years. This is not a criticism of religion - I am somewhat religious as well - but this is a feature of humanity. It would be incorrect to say that everyone who considers themselves insert religion here perfectly believes it. In fact, I would argue that there aren't many people who are 100% faithful to a religion (if you could even quantify that sort of thing). But one thing we can agree upon is that religion does bring people together.
In your case, the magic is real - divine beings are born every now and then. Even if the eldritch forces do not breach our reality ever again (even though the battles are constantly taking place), if taking the occasional divine child does not exert too much stress on society and to have such a child is culturally accepted as an honor, this belief will persist. In fact, in this case these demigods could end up becoming worshipped by the people of our plane, and this would vary from place to place depending on what these kids were like growing up (I'm assuming they never come back).
Will there be resentment? From some, yes, but in our real world there also is, but this hasn't effectively killed many belief system - interestingly, some beliefs go out of fashion when impractical, regardless of whether there was any resentment towards the belief.
Make it ceremonial
I kind of hinted at this idea already, but make the act of having a baby demigod a mark of honor. I understand that this isn't obvious until later, but when such a person is found, make the demigod recruiters have a ceremony to properly induct this new recruit. In a way you can think of this like how in parts of Southeast Asia boys become monks for a part of their lives - or even all of it. It is not seen as a punishment - none of us would think so, and many of them become monks despite not being 'forced' into it.
The most important part of the ceremony: make everyone see it! If this ceremony is done in accordance to a primary or local religion, it holds fast better.
Visitation rights + Retirement
Make these demigods come back every now and then. Allow them visitation back to our own reality, and even retirement. I understand that this is not the Jedi way, but we know what happened to them. The Sith got it wrong too, by being overly passionate, so it would be best if these demigods got to go home and help out their families and communities. To make this more real, give many of these demigods some kind of PTSD. Give them bizarre scars too - nothing too graphic, but enough to prove that something is happening in the other realm. Even if they won't go mad like normal people, what kind of person can fight eldritch horrors and come back the same? What if they lost loved ones on the line of combat? Make sure they keep coming back and keep telling their stories.
Let the older warriors stay back and exist in society in constructive ways, like teachers, administrators, and more. Let them use their powers (if they have any left by then) for the good of humanity directly - not in combat. You can make retirement optional, but provide the option. As populations will grow, the number of divine children will grow, and the ratio of divine:normal humans will remain the same. If a significant number of these divine children come back and take up helpful roles in society (not as warlords or anything) maybe people will continue to respect them.
Lie + Remove Exploding Teenagers
You pointed out that these kids will explode if not transformed into proper demigods. Make this organization a community that helps these kids in need, or have them interpreted like the body-carriers in Black Death - a terrible but necessary job in society. Make everyone remember why these kids get removed, because even if everyone believes the demigods, the demigods are likely to still miss a child every now and again, causing explosions. Even if they don't believe, they would be pretty big on not having their village destroyed because some kid hit puberty. You can add a coming-of-age ceremony here, with those 'passing' it being the kids who don't show signs of exploding in the future.
These special kids will be hated by at least certain cultures, and could make some demigods unwilling to fight harder against the eldritch forces, as humanity had been cruel to them in their memory, adding additional complexity to different characters.
Keep in mind, all of these ideas will only keep interest in this effort for so long. There will always be people who think these guys are a remnant of the ancient times, even if they aren't baby-snatchers or anything. In fact, there will even be people who hope the eldritch forces eventually win. But don't make these beliefs uniform and a defining point for all of humanity. If you do pick any of these ideas, don't work with that for all humans and their communities. Europe is a pretty small continent, but even they have a variety of cultures living near one another. Make sure different cultures exist in the human world and make their treatment of these demigods differ in different ways.
I hope this helps.