In order for magic to manifest in this world, one has to recite a specific incantation following a visualization of the anticipated result, and only then will the magic occur. Amazingly, many students from the special school of the disabilities, especially the mutes, seem to be able to perform magic unaided. This appears illogical because, in order for magic to be performed successfully, one must not only speak intelligibly but the pronunciation must also be accurate! Since the invention of voice synthesizer and electronic recorder didn't happen until much later, how do the mutes cast magic spells which require an incantation?
Just as blind people can be given a well-trained guide dog, those who cannot speak can be given a trained talking raven.
In your scenario, this type of raven has been bred over centuries to be the best possible of mimics. They are trained from an early age to recite incantations in response to hand signals.
The requirement of magic is that the voice must come from close proximity to the thought. The raven perches on the mute mage's shoulder and that is just as close as the person's vocal chords are or would be.
Even ordinary ravens are capable of this to some extent. Just imagine how much better they would be after generations of selective breeding and maybe even a little magical enhancement.
Ravens can talk! (video)
I guess the Pied Piper of Hamelin came out of this school.
The trick is being able to find a translation of the spell which works well when it is casted by playing the appropriate sequence of notes in a musical instrument, be it a piper, a flute, a harp or any other instrument with a sufficient extension.
Once the key is properly played, the spell is activated.
Words aren't important
It was never about the incantations but the mindset formed saying the words.
In the Marvel Movie Doctor Strange, Strange says to the Ancient One he can't cast due to the injuries to his hands at which the Ancient one shows him a caster missing a hand completely.
Magic is all in the mind. Incantations, mystical ingredients, hand movements, artifacts all do nothing except assist the person to reach the correct state of mind to cast the spell exactly like Dumbo has a "magic" feather that helps him fly.
Nobody realized it is not the sound that is important, but the magic is gesture based - and specifically, the Magic Recognition System is tuned to very accurate movements of your lips and tongue (who said gestures have to be performed by hands only?). For the speaking folk, the "precise pronunciation" is just a side effect of this precise mouth movement. And as everyone who learns foreign languages knows, phonemes you learn in your childhood influence very much the set of phonemes you are able to reproduce accurately (AKA "having an accent"). Thus assuming you learn the magic in your adulthood, the speaking folk has quite a trouble mastering weird and quickly changing mouth shapes, but the mutes are not hindered by their ingrained language (unless they are mute because of a physical deformity, like missing tongue or equivalent).
Sign language is as legitimate a language as any spoken one. The same parts of the brain are used in the "recitation", and that's all that's necessary to cast the spell?
And, yes, there's no reason sign languages couldn't have existed in the Middle Ages. Deaf communities naturally create them ex nihilo if necessary.
Since the invention of voice synthesizer and electronic recorder didn't happen until much later
As a general world building rule, in any system where you have magic but lack a technology that would be helpful to your plot, you can usually find ways to replace the technology with a magical equivalent.
Need to fly but don't have planes? Use a flying carpet.
Need long range communication, but don't have phones? Use a palantír.
Need an army of killer robots, but don't have robots? Build constructs.
So, just having a lack of technology does not really preclude your setting from having any modern wonder you want, including voice synthesizers.
To resolve this, have mages who are able to speak use their magic in lue of technology to record their voice performing incantations into magical stones, scrolls, wands, or what ever best fits your setting. Then when it comes time to cast a spell, the mute mage need only tap the enchanted object, and it will recite the spell while he performs the necessary gestures.
A Possible Frame Challenge: If your setting has healing or transfiguration spells, you will want to address the elephant in the room about why you would ever have a disabled mage to begin with. If a disabled mage gets to the point where he is casting magic, his first priority will probably be correcting his maledy; so, the very concept of the Special School of the Disabilities may actually be flawed unless you put some very clear limitations on what magic can do in your world.
There's a common trope in fiction about someone having their tongue cut out to prevent them being able to talk. Shakespeare used it, and he got it from Ovid, who probably got it from somewhere else. Of course you'll find it in Game of Thrones as well.
And it's entirely factually incorrect. I have personally met a girl who had no tongue, and she could talk intelligibly. It took a bit of work to "recalibrate", but it was doable. Her speech was rather similar to that of some people with cerebral palsy, who may not have the necessary control over their tongue for fully "normal" speech.
The sounds we produce in language come from breath control, from the vocal cords, from the tongue, and from the lips. The loss of any one will certainly be an obstacle to fully "normal" speech, but it need not prevent it.
If you eliminate the tongue, you still have access to all vowels, to the "plosive" letters B, M and P, and to the "fricative" letters F and V (and S and SH by using teeth instead of tongue to restrict airflow). With a bit of work, CH sounds are OK too, and sharp exhalations (using breath control) can fill in a bit for some other sounds. If your incantations can be limited to those sounds, there's no reason someone without a tongue couldn't still perform perfectly well.
The loss of lips would limit different sounds. In this case the "plosive" letters are lost, and some "fricative" letters. Teeth can still substitute to some extent though, and mouth shape is still enough to produce vowels reasonably effectively. If your incantations are all about vowel sounds, that will still work perfectly well.
The loss of vocal cords will naturally reduce the volume - but whispering is (basically by definition) talking without engaging your vocal cords, and we all know that works. Some letters require vocal cords to be produced - the difference between "f" and "v", or "s" and "z", is down to your vocal cords. If your incantations don't need those sounds, or those sounds don't need to be distinct, they will still work. This would also be a clue that only a low minimum level of volume is necessary for an incantation - Gandalf could whisper "you shall not pass" and it would be just as effective as declaiming it forcefully.
Basically, only the complete inability to consciously control breathing would prevent any speech. Anything else, people can deal with to some extent; and then it simply depends on whether what's lost in translation is important for your magic system or not.
Finally we also need to consider selective mutism. People may be unable to talk in some settings, but have no problems in others. Someone with selective mutism may be unable to talk to other people, but be perfectly able to voice incantations.
Mute people are generally able to whistle. They may use a whistling language such as Silbo Gomero in lieu of a regular spoken language to form the incantation of a spell.
Alternatively, they may use pasgas™ for the incantations.
Finally, it may be that the visualization is what is important. As you visualize, you must also hear the incantations with the "ears of the mind", much like you can recall and "listen" to a song inside your mind. Speaking or singing an incantation is just a surefire way to get the proper incantation in your mind. Mute casters have trained so as not to need to speak, they just hear it inside their heads and thus are able to cast successfully.
Wikipedia tells me in the Tibetan Buddhist tradition, spinning wheels with prayers written on them has much the same effect as orally reciting the prayers.
Perhaps mutes have discovered something equivalent to speaking the incantation that doesn't need them to speak.
Perhaps the technique is kept secret by mutes, out of solidarity: All their friends from the school for mutes are keeping themselves employed as spellcasters using the secret technique, and revealing it would leave them all unemployed.
Or maybe the technique is something well known, but only available to mutes - maybe making your vocal chords resonate with the magical aether allows you to subvocalise your incantations, but you've got to go 90 days without speaking to achieve that resonance.
Or you could make mutes' extra ability religious in origin - sure, in our world monastic vows of silence are mostly about quiet contemplation and even the quietest religious orders talk some of the time. But maybe in your world there's a god-like power who just really likes silence, even without worship of contemplation. It could even be a mysterious forgotten god, and mutes have satisfied it by coincidence.
Language in all it's forms
There are two requirements: incantation and visualisation.
An incantation is a series of words that form a spell. An incantation is thus just as valid on paper as spoken. That means the incantation can be done in any form that represent those words. On paper this can still be difficult, as some of the subtleties have to be described instead of written down ("It's leviosa, not leviosa). Still possible if you devote some more time to the written language, but difficult. Some forms like sign language could add these subtleties in each movement of the hand for example. So any broadcasting of the incantation of any form of language is valid!
Frame Challenge: Translators May Be Necessary
If you decide to make magic more mental than physical, Thorne's answer is correct. If music, ravens, or gestures could do the trick, those respective answers will be correct. However, you said one has to recite the incantation but mutes are casting regardless, seemingly without aid.
Well, guess what? The term "mute" today is specifically applied to a person unable to use articulate language due to deafness. And deaf people can speak, it's just very difficult to learn how to make the sounds and know what you're saying when you can't hear. In other words, deaf people should be able to chant and therefore cast just fine, albeit with more effort.
Alternatively, if by "mute" you mean anyone who through a mishap or genetic disorder cannot speak, being mute does not necessarily mean being unable to create sound. Think about it; breathing in and out makes a sound now doesn't it? And whistling doesn't require vocal cords at all! Therefore, mutes can create a language of their own based off those sounds and cast with it.
Breathing could be used in a sort of Morse Code, with short, forceful puffs for dots and longer, drawn-out breaths in between. Whistling could be used to sing a tune, and if the tune corresponded to musical notation for a spell, it could count as an incantation. Or you could have a translator.
Seriously, what's wrong with a translator? In Harry Potter, one of the most popular stories about mages, it's technically possible to cast without a wand, but few people ever do because it's so very difficult. Perhaps most mutes are the same; they need a translator to recite the incantation they're signing, but they also know some mutes, through extensive training and sheer force of will, can cast with only their thoughts.
The fact that the mutes are seemingly unaided could come from a telepathic link; if magic is an expression of who you are (just like your unique way of speaking), then translating for someone could forge a powerful bond between the translator and the speaker. Basically, when the mute wants to cast, the translator senses their intent and speaks for them.
You may not like this idea, but it has so much story potential. Perhaps a mute-translator pair casts stronger spells naturally, because both are contributing their will and/or mana to the casting, or perhaps they share the cost of spellcasting with each other. It could even be both!
This mute-translator pairing could also serve as a check on powerful mages; just as the translator gives voice to a mute's will to enable their casting, a mute can do the reverse (give silence in the place of speech), therefore preventing their translator from casting. All in all, this was a very interesting question and I hope my ideas help you.
Some people lack the ability to speak due to paralysis of the larynx. They cannot send nerve signals to make themselves speak.
However, a magical spell can be cast on them to force them to speak, whether they like it or not, operating their muscles according to that caster's dictate. That spell can even be cast in advance, to force them to speak the incantation when they make a specified gesture.
The dark practitioners of that grim school of thaumaturgy are more than eager to demonstrate that they can transform the lives of the disabled, just as designers of brain-computer interfaces wax enthusiastically of their plans to improve the lives of paralyzed veterans.
A common concept in Magic is the use of familiars - an animal with which the caster has an extra-sensory connection. Using the Raven example from chasly's answer, if a mute person has the aptitude for magic, they are paired with a familiar capable of speech. They then use their extrasensory (psychic) link to speak through the familiar. This provides the full combination of framing the thoughts in their minds, exercising the power of their will, and voicing them aloud (through the familiar), and leaves their hands free for any gestures that may also be required.
What Is Your Magic Based On?
- Do you have to be born with a gift (like the wizards in Harry Potter)?
- Can everybody learn magic? (Maybe with a little natural talent involved, like music or complicated mathematics or any craft; a simple fire spell might be as common as matches!)
- Maybe there are different kinds of magic in different parts of the world, either forgotten, banned, not yet detected or rejected by other societies.
In Harry Potter house elves use a different kind of magic to which different rules apply, in The Last Airbender the different elements are controlled in different ways.
This might affect, if the spell just helps the magic user to focus and channel his own magic or if some sort of articulation is needed.
Another big question is, if you need to simply visualize / articulate the content of the spell somehow, or if you need actual actual words/language.
As L.Dutch already mentioned, there are more forms to express yourself in detail, without language...
Music lets you feel the emotions and images the artist wants to convey.
In Asia and the arabic world calligraphy is an eminently respected craft. At first glance you might want to distinguish the spoken / performed magic from the magic texts. On the other hand there is a long cultural tradition of belief that writing holds magical powers, from ancient Egypt to the Great Plague of London in the 17th century when the spell ABRACADABRA was used as a safety measure. And (written) magic symbols, sigils and summoning circles are commonly used in fantasy.
Different articulations to perform magic might be distinct to different cultures, with native tribes using ritual dances (e. g. rain dance), while master craftsmen are able to incorporate simple blessings or defense magic into paintings or temple guardian statues and lucky charms.
Developing that concept further, every art form and every means of communication holds the potential to envoke magic, however, over time rituals and spells have been passed through by famous teachers, setting the foundations for the different magic traditions in your world, as manyfold as those in our very own.
Aside from the mutism aspect it makes a huge difference in general,
- if it is only important to understand what you are saying
(every mage casts in their mother tongue),
- if there is only one magic language (like in Eragon) and/or
- incantations work, even if you don't understand the language you use
(like the Sectumsempra spell in Harry Potter VII).
Another question is, how intricate spells are: Just controlling/summoning something by one word ("fire!") and the thoughts on how you want to use it shape its form (a wall or peak or blade of fire) or does the magic effect have to be described more detailled or with stipulated hexes ("giant wall of flames!").
Magic users could invent their own languages, making it impossible for enemies to foresee what attack to parry. They even might develop a language that use the same words of an existing language but with switched meanings (saying "fire" and creating a blizzard). Or simply thinking outside the box and repurposing magic spells in a way an observer can't predict the outcome (casting "water" to stop the bloodflow of the enemy's body).
Mute Mages vs. Spell Casters
Verbal magic might be easier and safer but also limits the user to the spells he has learned and especially how he learned to apply them, making it harder to think out of the box, while nonverbal magic is much harder to master but allows a more intuitive and free approach. A spell caster casts fire to burn an enemy, while a mute magican just disintegrates the enemy (Thanos style). A spell caster either uses the spells for "rock" or "fire" to reshape a boulder while a mute mage just reimagines the outcome. Therefore maybe mute people might be less likely to master magic but those who do are outstanding.
It could be possible, that verbal and nonverbal magic have different weaknesses and strenghts, e. g. non-verbal magic being better suited for creating, while verbal magic allows quicker fighting.
Also possible: Not being able to speak generally elevates the magical senses (a disabilitiy leading to other heightened abilities).
A Question of Society
There can be reasons why non-verbal magic is not accepted in society although not being much different in effect. No world is perfect and each has its own forms of exclusion, injustice and thoughts of surpremacy, cf. the historic role of women or white surpremacy.
- Words stand for control, order and tradition.
The first known grammar was an attempt to preserve the original language of holy texts. A quasi-religious, quasi-military magic order could be interested in maintaining the sacred order of yore.
- It is also a question of teaching methods and educational understanding:
Impaired children who keep their costudents back, or children who need special support to unlock their potential.
- And, as Willeke mentions below, historically mute people were taken for stupid.
(Btw. I didn't think of that argument myself but didn't want to omit a good point after I read it)
- Magic that solely rely on focus and willpower is more dangerous (imagine thinking a death curse while suddenly being distracted by your hot spouse).
- The use of magic could just be deemed unthinkable for mute people in a society centerd around magic spells, just think of the role of women in religion, society and war over the last couple thousand years. There might even have been some famous exceptions without changing the overall view of the people (cf. historic exceptions of female leaders and warriors).
- Here, mutism might be seen as a large humiliation.
- There was the thinking, that the appearance of a person matches their character - someone hidious must have a terrible character, deseases and desasters are a punishment by God and humans are not allowed to interfere with divine will and fate. Lore has it, if a child with magic abilities is born mute, this is divine intervention to prevent the world from a powerful evil wizard.
- Verbal magic could be part of an honor code: The performer is revealing themself and enabling the opponent to react, similar to the regulation of concealed weapons or the Geneva Conventions or the difference between a honorable duel an a dastard murder. This also proves a magic action to be a willing performance, not just a lack of control over one's magic powers (cf. obscurus in Harry Potter).
- Verbal magic might be considered more civilized in comparison to the wild tribes performing intuitive magic (cf. colonial Europe).
- There might be a rivaling empire with very different opinions (good and bad), which, besides commonly accepting non-verbal magic, commits unthinkable actions like human sacrifices. Therefore non-verbal magic is connected to the evil and wrong practices of the enemy.
To sum up:
Mute children require more effort (a personal tutor, ideally with the same handicap, as the academy refuses mute acolytes) by the parents who are ashamed and disappointed by their offspring and underestimate them. Furthermore it is a sacrilege, according to society, religion and tradition. So how often do children actually find their way into the special school for unusual circumstances? How widespread is the word of that institution at all? Even if mutism doesn't make that much of a difference on a magical level.
This might change, if e. g. a war or a loss of mages boosts the need for more, through a social revolution or a mute hero or royal successor. However there is no guarantee.
Not sure if you find this applicable, but in Dungeons & Dragons, there's a metamagic feat called "Subtle Spell" which allows the caster to disguise material, verbal or somatic component of the casting (or was it all three at once?), so technically a mute character could cast the spells non-verbally with this feat.
Its not the sound that actually matters, but instead the mouth movements.
In your case the magic is drawn out of the body when you breath. This magic is without an purpose so just "sits" near the breather who then breaths it back in in the next breath. However, if you make the right mouth movements (which in a normal person just so happens to also be recognizable speech) the expelled magic now knows what to do and goes and does something useful.
A friendly wizard visited the school for children with special needs, and cast spells to make their lives easier. The boy with no feet was blessed with the power to levitate, the boy with no eyes was given an ugly and slightly mischievous imp that whispers in his ear what it sees, and the girl with no voice was inflicted with a curse that makes her innermost thoughts echo audibly around the room.
The teachers were annoyed by these disruptive gifts and did not invite the wizard back the next year.
To answer this question you must first define how the voice act in the casting of the spell. What part of the "saying a spell out loud" enable the spell to be cast? Is it the vibration that get created in the air? If so, is it possible to reproduce the spell by achieving similar vibrations with instruments or props? Or maybe the sound must be produced by the caster himself, because it need to be "imprinted by his personality/spirit/whatever". If so, does the sound has to come from the vocal cords of a person, or can be generated by whistling/clapping/whatever? Or is not the sound the important part, but the fact that by pronouncing it you are less prone to mistaking your intention with something else (If you have read Eragon this may sound familiar)? In this case with adequate training you may be able to do it.
You mentioned voice recorders. If someone is able to cast a spell by using a voice recorder, how is he able to do that? Is the sound (vibration?), is the "intention" to cast the spell being recorded with the voice?
Answering this questions may help you come up with an answer.
Posted as a comment but I am asked to post it as an answer:
As far as I understood, mute people in medieval times were seen as 'unteachable' and 'dumb' in the sense of having no intelligence.
So if someone was born without the option to learn to speak, it was very likely that they would never learn enough to need those incantations.
Having lost speech as an adult, they would likely face a lot of problems and not being able to do magic would only be a small one.
Added to fill out the answer:
In our current western world people with handicaps are seen as fully able people who have some restrictions. But in big parts of the world today the situation is not as good. And if you go back in our history, even 100 years ago people with handicaps had it a lot worse. Not being seen as fully human is one thing, seen as valueless as a mongrel dog that has lost it usefulness was more likely in many cases.
Go back 3 or more centuries, forget special education. In fact, most people did not get any education and those who were not fully able to keep up with the brightest would not get any education if they were not the children of the very rich.
And even the rich would often not admit that they had a child less than perfect.
Of course as a writer you set your own world, you may be able to make a setting in which a person without speech can function as a full human, but it will not be as it was in our history.