Most of the solutions provided aren't particularly good.
The reason why is because in our world we actually did have a place and period where "Art" was widely available and practiced in a feudal society. I'm not sure if the parallel is intentional or not, but reading the "body mana", my first instinct is to shorten it to Chi. And of course, the connection between a martial art as per the title and a magical art that controls a bodily flow of energy, I can't help but think of Karate, Kung Fu, and other forms of martial arts from that region.
Think about it. For starters, even if we don't have magic in our world, as far as feudal (and many modern) east asians were concerned, it's real, at least in their minds, they think it's real and has the effects you describe. Oriental martial arts have a very strong spiritual component where practitioners try to control the mana in their bodies through breathing, body motions, conditioning, meditation etc etc. And the results are there to some degree. The dynamic meditation aspect of an oriental martial art helps people relieve stress, it's physical exercise, there's often a focus on having proper posture, and on being good for back and joints (again, whether true or not, most people from the region believed it), so obviously a practitioner would have good health, and based on our stereotypes, a lot of people in the Western culture think of east Asians as being very healthy.
A lot of concepts taught in oriental martial arts are about using biomechanics to amplify existing strength, and I've seen it in practice. My teacher, a thin 60 year old gentleman of about average or slightly less height, manages to throw around men twice his size and half his age with ease, due to technique and instinctive understanding of anatomy (and they're not just playing along, I've tried resisting and he knows how to deal with that). So there is the strength aspect.
And like I mentioned before, there is a great emphasis on spiritual and mental development through meditation and similar practices. A calm mind is a mind that can see things coming more easily, can anticipate an opponent's moves, notice patterns. There's the enhanced reflexes aspect.
So all of the reasons regarding time, cost, literacy, etc, aren't compelling reasons because in my opinion, most people familiar with oriental martial arts, Avatar: The Last Airbender or generally has been exposed to the waves of fetishization of east asian culture, would be able to recognize the parallels. Somehow, in our world a rather large region had peasants practicing and having widespread access to (what they believed to be) an Art similar to what you describe. I'm not saying every Asian peasant knew kung fu and could decapitate someone with a skillful toss of a straw hat - but it was a common thing for people to practice. Or at least, it was not uncommon. Karate specifically was developed as a response to a weapons ban, so peasants can figure out a thing or two on their own. Especially if the Art is more pronounced than the biomechanical benefits I outlined above, practitioners would be able to further refine it and spread it through trial and error (due to having much more noticeable feedback).
It might sound like a long winded way to say that "your premise doesn't have a way to work the way you intend it to". And to be honest, when I sat down to write, that was going to be my conclusion. However, in the process of writing, I did have several thoughts occur to me. While martial arts were developed and practiced all over the world, and certainly there's no shortage of spiritualizing and ritualizing martial arts, the east Asian martial arts are unique in just how intertwined they are in religious/spiritual aspects (and they do have their origin in a form of meditation practiced by Buddhist monks). The more important point is that the rest of the world didn't develop the same thing despite the benefits I outlined above. Even if the benefits are just in the mind and not an actuality, the east asians were not uniquely superstitious.
However, there is a very simple reason why, for example, European martial arts, while rich in history and technique, don't have the same dimension and focus on cultivating chi or mana. And the reason is religion (and to some degree culture). While the buddhist/taoist/confuscian (and in Japan, shintoist) mix of worldviews makes trying to fight using chi a very natural concept in the east, in Europe, the influence of the Christian Church makes it very natural that such an Art can't take off. I know of two influential modern traditional Christian priests who outright condemn Karate, Kung Fu and Yoga as a form of sorcery and devilry (that would be Fr Chad Rippiger, an influential Catholic exorcist, and soon-to-be-declared-an-Orthodox-saint Fr Daniil Sysoev). Rippiger claims to have exorcised people who were influenced by demons for practicing such things, if I recall correctly. While opinions of a couple traditionalist priests doesn't make for dogma, you can imagine that despite the Spanish Inquisition being 99% Myth (see Wikipedia entry for Black Legend of Spain, or the BBC documentary), such an Art as you imagine wouldn't be very welcome in a feudal europe.
So that's one venue to go with. For whatever reason, the religion or culture of the common people makes them afraid or distrustful of using magic. They believe it corrupts the soul. Or maybe they see how the people who do use the Art become progressively bigger jerks (nobles are jerks already, and it's not uncommon for people to get prideful as they become better at something. Think of how many stereotypical martial arts teachers act like insufferable snobs that beg for a smacking in various films). It could be that the aristocrats are more pragmatic, cynical, or educated, and that they don't share the same apprehension. It could also be that the nobles and the peasants are of different cultures, which is not unheard of. Ireland and Norway did not have nobility from their own cultures for a very long time, due to being ruled by England and Sweden respectively. Or England itself being ruled by Norman invaders. Perhaps in the region where your story takes place, the Art using nobles (who had a great advantage from using the Art) took over the region and subjugated the local population, who are resentful and mistrustful and don't want to engage in practices the nobles do, out of fear of being assimilated and having their own culture wiped out.
Another possible solution (again from real world parallels, this time of DEMONS :) ), since there are magical creatures, it could be that there are certain creatures which are attracted to hunt Art users. Perhaps one's mana grows and develops through the use Art, which creates a mana spike, making the creatures that normally ignore humans, hunt them. Nobles would be living in castles, and have their bodyguards, and have much more advanced techniques. But the peasants who do practice the Art and live in (sparsely populated) rural areas make themselves a very tasty and easy target. Perhaps the nobles have the means and the resources necessary to ward off the creatures, or mask their enhanced mana signature. Or, if mages use external mana, an Art user who cultivates a greater bodily mana, becomes a sort of walking battery or conduit for them. If society gets to a point where Art users are common enough that mages can use criminals and undesirables, then there would not be such fear, but it would be a bumpy road since it would be the best of the best who pioneer the path.