A Modified Scold’s Bridle

Scold’s Bridles, also called witch’s bridles, were cruel punishment/torture devices used on mostly on women, originating in the 16th century. Please note that I do not condone the use of such devices, even in history; I am simply explaining their usage as a sort of illustration. They were publicly humiliating masks the people had to wear, due to being suspected witches because of ‘scolding’ in public, or ‘gossiping’ and ‘rioting’. A spike embedded into a ball near the tongue would pierce into the tongue when the wearer attempts to speak or eat, making such actions impossible. It also could be a metal piece pressing the tongue down instead, causing it to be very difficult to speak coherently.
The Virtue Mask
It is difficult to imagine joining metal to the face as there are frankly too many biological consequences, but instead your mask can be a sort of scold’s bridle. These masks have the advantage of being able to be put on whenever one joins the religious order. There would be spikes on the back of the mask, which would not hurt the user when they put it on, but when one tries to remove it, spikes pierce into the back of the head, so effectively, it stays on. In mask form, it would look similar to the following, but have spikes that extend the opposite way, for when the user tries to remove it (from JoJo’s Bizarre Adventure):

The public can see how the mask is attached, so the mask is like a symbol of virtue, representing a sort of lifetime dedication to their cause, rather than the humiliation associated with a Scold’s. Just please change the mask design into something more plain, otherwise the public would be absolutely frightened instead.
If spikes are too much, instead, the mask can be two metal halves (like a helmet), and be melded together after being placed around a person’s face. Or, it could be a mask welded with metal straps on the back of the head.
The mask should be slightly larger than the person’s head, to prevent the welding heat from searing the person’s face forever, or it could be nearly face fitting searing a vertical scar into the side of their head forever as it is welded on (shudders). If your story is to feature deserters often, and the religion is extremist even in extremist religions, I would say the form fitting mask would be ideal. Otherwise, the large mask (more like a helmet) may be nice, serving as a symbol of the religion.