No Muscles. No Problem.
Do not worry. You can safely make your Medieval women strong as Medieval men. They will not become bulbous and veiny like Goku from Dragonball Z. Because Medieval men were not bulbous and veiny like Goku from Dragonball Z.
Medieval soldiers were not supermen. Most of them were farmers. Medieval farmers worked 8-12 hours a day and had great strength and endurance. But they did not look like muscle-bound super-studs.
Medieval farmers are extinct. The closest we can find today is people living in the developed world who do manual labour all day. The above men are described as Mali Tribesmen from Orissa, India. Take the above with a grain of salt, since people who do purely manual labour are rare these days. And "tribal tourism" is certainly a big moneymaker.
Also check out this guy who spent some time with the Hadza hunter-gatherers in Tanzania. Again I cannot promise you these people live this way on a daily basis. But Wikipedia at least suggests there are only about 1200 Hadza people. So perhaps they are the rare exception.
Also check out any tribal people from the Amazon. You cannot be sure how these people live when the camera is turned off. But if they live in deep jungle then you can tell they don't have cars or mechanized farming. So they do everything by hand.
Dressed differently, the Mali man on the left would be invisible walking down a busy city street in the West. Until you shake his hand and notice he is made of solid iron.
None of these people look like bulky superheroes. They do not have stupid well-defined muscles like Maki Oze from Fire Force:
A real-life Maki Oze could bench-press more than the average Medieval peasant. But only because she practices bench-pressing every day. The peasant could probably outlast Maki if they were lifting a smaller load. And Maki would certainly crumble if forced to push a plough for 12 hours straight.
For a modern example of what a woman looks like, when she trains every part of her body, without focusing on appearance, check out olympic climbers. Without her clothes on Shauna Coxey looks like this:
and not like this:
You can see the muscles, but would not suspect she is one of the strongest women on the planet. But check her out on the wall. And look at how strong she is