Short version of solution : Mid Ranking Management.
Long version : lots of things I don't like.
In my world, society has progressed to 2050-ish, a new nation rose on Greenland led by an aristocratic community called the "Grey".
Or something more in keeping with the national language and culture, perhaps. "The Grey" seems a rather uninspiring name for a group leading anyone. It conveys no meaning or association with any historically or socially important ideal. It may be the sort of name an author might pick, but it's not going to help get you to power and at some point you have to start not in power and rise to power.
"The Grey" is a dark name and would convey, if anything, a sense that they might be dangerous. It's not an accident that groups tend to go for names like "New Dawn", "Shining Path and so on. Positive spin.
Rethink the name.
Numbering at 1000 or so
So barely enough to populate even the most senior positions in an administration.
, the grey is a group of highly intelligent
In what way ?
Intelligence can take many forms and if they're just randomly intelligent and individual members can funnel their energies into medicine, art, music, literature, games, programming, engineering and so on, that won't really give you many of the 1000 who are actually skilled at the skills of politics and economics that they'll actually desperately need as a small group to survive.
, genetically enhanced (with slight improvement in brain power and longevity)
Slight ? So if I concentrate hard I can out think them ? So the top 1% of the million or more population can match them or better them ?
They'd better be well beyond "slight".
Longevity isn't much use if you're a minority group that can find itself shot against a wall in a revolution.
So you want long lived individuals or a long lived clan ?
Dogma rears it head.
Dogma is a weakness. Any form of religion is, by definition, a factor that will be considered reactionary by people outside that religion.
All a religion does is make it focus for resentment. "Why are we being led by this minority ?" is going to be followed, eventually by "Hang the Bar Stewards !". History is pretty clear on this point.
If you have a religion and you want to hold power, it must be a religion the ordinary populace are comfortable with.
Which leads us to ...
Details about their religion is unimportant,
Yes, they are. Details of a practically minute nature are responsible for some of the bloodiest and most ruthlessly pursued wars in history. The devil is in the detail.
but it is not at odds with science/progress,
It must be. Religion requires belief without evidence. There will always be a point where this clashes with science, which require no belief unless there's evidence.
There will always be a point of contention, usually many, between these two fundamentally different outlooks.
and its overall effect is that, among true believers,
Question : how do you tell a true believer from someone who just claims they are ?
Answer : tell them you'll spare the ones who relinquish their faith and burn the believers are the stake.
A system, I might add, which has been used extensively by humans and, hard as it is to believe, the true believers often will go to the stake and you're left with the cynics to feed.
Another way of viewing this is Who Polices The Police ? Who says you're a true believer and how do you prove it ?
this religion inspires a sense of duty,
One man's duty is another man's holocaust. Dogma is a problem and, again, who decides what's duty and what's unworthy personal obsession ?
and gives the Grey a condescending attitude towards nearly all outsiders, seeing them as hedonistic, wasteful primitives.
Surely they'd start seeing each other that way as well ?
Normal greed, envy, lust and all those other human emotions will lead you to look at not just your servant, but your master, as a waste of space you could be using better. :-)
Also, the Grey exercise their power by collecting a small percentage of national tax (which they share among themselves from a common pool),
Bet that's popular with the vast majority they depend on to not rush teh palace gates !
directly controlling the army
1000 people don't control an army. They control, at best, 1000 people.
The army will be the de facto rulers and, as history proves, if you depend on the army to keep you in power, you're just a moving target.
Something other than a terrible name and a condescending manner is going to be required to stop all these armed and highly trained people from just turning on you.
and hold veto rights over the more "normal" political activity performed by a mostly democratic government.
A mob also holds a veto right. Usually in both hands.
A minority religious caste creaming money off the top and who cannot be democratically removed is, in no way, going to successfully pass itself or it's society off as a "mostly democratic" government.
If it looks like a nobility, acts like a nobility and swims like a nobility in it's private swimming pool, it's a nobility.
When the Grey replaces dead members, they prefer using their own children (which they produce with calculated care) however, it is possible that the children they raise are not as religious as their parents or are unwilling to act as rulers.
So not produced with much calculating care at all.
This means that either many children have to be produced so that some may suffice for future leadership, and it also generates the problem with the under-performing children, who may become a factor of instability.
Actually just the ambition of the ones being groomed for future leadership is sufficient to create a lot more instability than anything else.
The ones who don't have ambition and under-perform are, by definition, no threat at all to the ones who have it and do perform.
They are a non-problem who'll be happy to get by on a basic "better than average" job with enough trappings of power to not need real power. Perfect mid ranking managers could hardly be described better.
If it is necessary to replenish the Grey's members by assimilating outsiders, the group may compromise its ideological/genetic purity and the sense of separation from the rest.
That'd be the same sense of separation that will find them waiting for execution after the revolution, I take it. :-)
It is also difficult to test the new members for their loyalty, more specifically, it's difficult to tell whether the new members are truly assimilated with the Grey's ideology or simply pretending to be so as to secure power.
As I've pointed out, it's just as difficult to tell what's really going on inside the mind of all these condescending, ambitious people.
It's fundamentally impossible to avoid this problem and still have any free will in a group.
There is no perfect solution.
You need a system of exams, formal and legally binding (on both sides) to test for ability, inclination and deception.
How do you test for deception : you use good interrogation methods (and that's the talking and listening type, not the idiotic buckets of water nonsense). You use detectives to check things. You use, in fact, a good Intelligence service. The exams just test technical competence and knowledge. You'd probably also test for ability to lead - check out how the military do that, because if there's one sphere of human endeavor that requires a better than average success at finding good leaders, it's the military !
I am thinking that culling the Grey's own children for sufficient replacement will work, and the under-performing children can be put in a sort of "pacified confinement" which is equivalent to a highly comfortable prison where the inmates are stripped of all power and cut-off from communication with the outside world.
I'll say this again : mid ranking managers. Practically the same thing without the weird connotation of "culling" - an unfortunate choice of word.
Nevertheless, I am not confident that this will completely dispel the inmates' capacity as factors of instability, and perhaps a disproportionate number of Grey's children will voluntarily choose this lifestyle, forming a significant burden on the Grey.
Gotta have mid ranking managers with little or no real power. Planet's full of them and while they're a nuisance and get in the way really powerful people (like engineers) they're not going to be able to form and execute any plan, either singly or in a group, that could possibly destabilize anything.
No revolution was ever started by a mid ranking manager. Well, probably.