Sooo many answers.
First, you won't find a system that leaves medieval entirely intact, and that's fine. One of the jobs of good world building is to think through the implications of anything new you add to a setting and extrapolating how it affects your world. There are many things we can do to make the magic fit in 'nicer' with your world, but your still going to have to think through what things it changes.
Going along with that, remember that if you can think of it not only have people come up with it, others have come up with counters to it. If magic-portal invasion is a threat someone creates a spell to prevent creation of portals in your castle. If people start throwing lightning bolts around insulated armor with 'grounding boots' to direct the lightning away from the body becomes common. If someone uses magic for assassinations others create magic tracking spells to easily track the assassin down and punish them etc. Creating intelligent counters to uses you don't like is ultimately going to be required to some degree.
Likewise there will be some things you may not want to 'counter' in such a manner. Perhaps magic fireballs prove more effective then arrows for long range attacks, and you don't want to create an obvious counter, like anti-mage individuals responsible for blocking or stopping magic attacks in squads. Then you should run with this, what does this do to combat? maybe armor changes now that protection from arrows is pointless. Maybe lots of small squads that are more spread out are used to limit the harm a fireball can do. Maybe assassination of mages before major battles are common. Most extreme, maybe defending a ground becomes far easier then attacking and tactics more similar to the world wars, where machine guns likewise redefined the ability to charge a point guarded by someone with lethal ranged weapons, occur.
In short whatever system you create it will be your duty to sit down and think of exceptions or how magic changes. Almost anything you really dislike you can justify a reason why it won't work, and you should leave some changes in and explore how it changes your world. showing magic change the world makes the world feel more real, and so long as not done too drastically does not make the world feel unrelatable. Just remember you can't stop at one step. If magic changes element A of the world then ask how changing A will affect B and C, and how they in turn affect D etc. It's an iterative process that can go a long way to building your world and helping to make it feel unique and real. Not always easy, but still desirable.
Having said that, lets focus on your actual question, what can we do to limit the affect magic has on the world. Not remove it entirely, but lower how drastic the affect is.
Magic is new
In many ways this is the easiest option, magic is a recent development. People have only recently discovered the existence of magic, or magic has recently become more powerful due to some new discovery or change in some magical principle.
This works so well because the people of the world don't know how magic works yet then their culture wouldn't have changed to adapt to it. Remember cultural adaptation is slow, even once a change means that the world would make more sense doing X it can take as much of a generation for the culture to realize that they need to stop doing Y and do the more logical X. Thus a culture can still be 'stuck' in some non-magical view of the world even though magic has been around for a little while. The culture doesn't make sense given magic's existence, but not because you made a bad world, but because your believable world is just as slow and foolish about adapting to change as people usually are.
Conservation of Everything
If you want your system to work and be at all believable your are going to have to stick to conservation of mass/energy very hard. Any violation of these principles leads to some sort of exploit to break thermodynamics in a way that's going to radically change the world.
Of course, it's nearly impossible to have an interesting magic that doesn't violate one of these fundamental principles in some manner, and if you do it's likely a magic that is not very useful. However, there is some room to bend the rules, particularly in conservation of energy. It's okay if you get more energy out of a spell then you put in, hand wave it by saying it's being drawn from some external source, However, make sure that energy put in is still proportional to energy gotten back out. Your mage is going to exhaust it's magical reserves in some manner if it keeps producing energy out of no where, and will do so before the energy produced is too drastic. Furthermore, having much of the energy produced only stick around while the mage focuses on it, and disappear once the spell is ended/disrupted, further addresses this.
Conservation of mass is in many ways more important to conserve. If you can summon something out of nothing and make it stick around you can bend practically every rule out there. Generally speaking I suggest either not being able to create any mass, or if done it's something that lasts only as long as a spell is actively maintained (for instance, a shapeshifter can change their mass to change shape, but as soon as they stop focusing they change back to their original shape, and you can't cut gigantic slabs of meat off of someone shifted into an elephant and have them stick around once he changes back etc).
Long range communication is limited
One huge change on recent world is the presence of phone, tv, and internet making communication over long distances easy. This caused a huge change in our culture, if subtly. Most obviously it changes the "us vs them" mentality we use to have of certain groups by making it easier to speak with them. It also means spread of ideas and memes across the world.
The most obvious, but by no means only, affect rapid communication would have is to do away with a feudal system. Nobel's had near-absolute control of his personal lands in medieval times, This was necessary because it was too slow and difficult to appeal to a king or the kingdoms law as a peasant, someone local and reachable was required and he had to have the power to make decisions immediately. However, with rapid communication a kingdom can pass laws and enforce them easier across his entire nation, without worrying that a decision he makes about a problem 3 months ride to the east will be out of date by the time it arrives etc. Once a king/government can effectively rule over larger lands they would then be oppose to giving so much power/independence to smaller ruler bodies. In fact the USA, and many other countries, have shown a similar tendency to move towards centralized governments as rapid communication developed. Roughly speaking states of the USA are analogous to feudal lords of the past, while they are much lager and with less absolute power they still serve the goal of creating smaller ruling body designed to make rules designed to support their local needs that a larger government has difficulty regulating. With the development of rapid communication more power has been drifting towards federal government over the states with time. As people through of themselves less as "Marylander" or "Californian" and more as an "American", who happens to live in a specific state, they became more open to the idea of central government because they were less worried about defending their individual's state's sovereignty over trying to support America as a whole.
The peasants would also be less likely to abide with feudal rule, which placed them in a near slavery to enrich the coffers of nobles, if they had access to rapid communication of their own. With more access to learning that comes with rapid communication, the ability for peasants to communicate and discuss revolution without feeling isolated and powerless, and awareness of faults of nobility and/or how they are spending the wealth your making them all would incline peasants to fighting for more power. We've seen revolts all over the world linked to twitter, texting, and other rapid communication making it easier to organize the masses.
In short the more readily available communication the more the tendency towards larger centralized governments, the more available it is to the lower class the lower the tendency towards ridged social class systems or subjugation of the 'peasant' class. This is only one example of many things that rapid communication changes.
As such it's best to be careful with any form of rapid communication over a distance. It's so absurdly useful, and world changing, that it would require substantial world building to work around, and likely will not look like feudal/medieval period your use to.
This also requires caution about anything that could lead to rapid communication, even if that was not the intent of the magic in your own mind or it is difficult to do. For instance the ability to transport from A to B equates to rapid communication and thus could need restricting. If such a transport spell were difficult and costly enough it may be possible to keep without completely doing away with feudal system, so long as you accept that being a carrier will likely be a huge part of any mages job, and consider what rapid, but very limited, communication, used only for the most important messages, would do to your world. It's generally easier to forbid this entirely though.
Certain permanent transportation portals from point A to B, that are huge public works to produce and maintain and can't be easily created or moved, are more forgivable. You just have to work with the realization that A and B are effectively neighbors once such a portal is created, culture, memes, information, etc will pass rapidly between their borders and the culture between the two must be adjusted accordingly.
Be careful with spells that keep working long after your mage is done casting
In many cases any enchantment that lasts forever is bad, though in reality this is simply a side affect of conservation of energy, if I cast a spell once and it keeps doing something that takes energy to do then eventually I'm going to have produced a disproportionate amount of energy. Though you could get around this by having mages constantly 'refresh' a spell to put in new energy in it.
However, I would take this a step further then just that. You need to avoid magic-as-technology if your setting the world in the past era, and that means enchanted items that can serve as technology must be very carefully handled. A piece of magic may seem small, like making a small stone glow with it's own light; light doesn't take much energy after all. However, if it's simple then mages will likely start producing and selling them regularly, and now you have a flashlight/lamp item that is free for everyone to use. This will drastically change the pattern of daily life, if light is so easy to come by (fires and lamps have a more significant continual expense and inconsistent lighting which limited how they could be used). You could say the stones are hard enough to make that they aren't cheap, but that just means only your nobles get your technology, which furthers divide between noble and peasant but still results in a pretty different world.
My suggestion would be to be very careful with any spell that a mage can cast and then walk away and leave running once their no longer focusing on them. You can have some, but if you want to make a system that's easy to 'fit in' to your target timeline your probably want to stick mostly to one of three options.
Large, stationary, works that took extensive effort to create and are constantly requiring mages to revisit and refresh. You can make these rare enough and easily decide which work you do or don't want to exist by simply saying only certain types of spells are cost effective enough to be viable.
Meta-magical items. By this I mean items that mostly affect magic itself. Most obvious being items that detect magic, or block magic, or allow affecting magic. We have no history of magic so these sort of items aren't going to cause your world to feel less in keeping with the time period you came up with since they only affect something that is new to your world. They are also potentially useful for serving as magic counters (see below).
Very short lived enchantments. Anything that requires a mage to make and tends to degrade in it's power quickly can't serve the role of technology or automation since the spells collapse too quickly to constantly be replacing the magical technology for regular use. This gives a useful manner to have characters who aren't magic have access to something magical for a little while, by saying it's just recently been made for them. A standard trope is that sunlight or dawn refreshes magic, so spells that tend to break by the start of the next day, or degrade at start of day and likely will collapse after only a few days, could be fair game.
You can always add other items as needed for your world. But for any other enchantment your need to put serious thought into how it works and what affects it will have. In particular ask how hard it is to make and if it will become something every day people would have, or even 'just' all the nobles and rich will have, because if so it may serve as technology and your need to build your world around presumption that a group of people have it. You may have some 'wiggle room' for very specific types of magic if you claim they aren't useful in most cases, but justifying why some odd magic can be created as an enchantment but no other magic can gets to be difficult.
Counter Magic
Magic designed to counter undesirable magics is a very common trick to address some problems. If something is too game breaking have people go out of their way to create a magical counter to that problem. It may be a spell that stops specific spells from working in an area, or a spell to stop all magic from working.
Separately, a good counter to many game breaking magics can be simple rule of law. Maybe magic can let you get away with lots of things in the short term, but your going to be found out and punished quickly. For instance maybe there is a spell you can work to charm people into doing what you want, but it will rear off after some time and they will remember you worked it on them and report it to the magistrate who will have you arrested and harshly punished? Or maybe regular checks for mind-magic are done and if they 'detect' mind magic on you, meaning you have cast it recently, your are likewise punished even if they don't know how you used it. There are many similar options, but basically whatever magic you work can be identified and is so harshly punished that few are foolish enough to risk the punishment for the reward.
However, it may be even simpler, people engaging in standard practices or building in 'mundane' anti magic solutions to protect from particular threats. Perhaps running water interferes with magic and so to prevent shapeshifters from entering a village an aqueduct is built that runs along the perimeter of the town, ensuring anyone entering it will show as their normal shape? Maybe it's known vampires can't enter a building without an invitation so no one ever invites anyone in their house, instead if a door is open it's taken to mean someone won't be offended if you enter without invitation. Maybe a mage can set you on fire with their mind, so everyone knows to preemptively attack a mage if they start casting a spell without permission so they can never work their magic on you?
The point to all ideas is the same, people come up with solutions to stop mages from doing things they don't want done. At the most extreme make it the nature of the magic that it's usually easier to counter any spell then to cast it and your pretty much prevent most malicious uses of magic.
Bias against magic
This is, at best, a minor 'fix' to any magic system, but a bias against magic can partially help explain why a society insists on doing things the non-magical way even though magic would seem to be more logical. This could therefore help solve some questions of "wait, if magic does that why not use it for XYZ" with the solution of "people don't like using it that way"
This could be the whole "magic is evil" view, but it doesn't have to be so extreme. Perhaps over use of magic is seen as decadent/lazy so people do things the "hard" way because they want to prove their not lazy, even though they don't hate magic. Maybe magic is thought as something that everyone knows should be saved for some specific causes, as such it's considered wasteful using it for some other reason, even if it would make it easier.
One of the easiest biases to justify sticking into society, other then 'magic is evil', is a religious one. Perhaps their religion believes that magic is holy and so only priests should use it or it should only be used for 'holy' uses. Perhaps instead it's believed that hard work is part of our lot in life and using the great magic given by god to frivolously to make our life too easy is decadent or sacrilegious.
The great thing about a religious bias is that it's much easier to create a far more nuanced bias that specifically says that "x" is okay but "y" isn't then most biases would allow, based off of the holy text and a societies interpretation of it. As an example maybe their messiah had some manual labor job and so that particular job is thought of as following in the messiah's foot steps and thus holy, thus the reason people still do that one specific thing by hand when magic would do it easier while still using magic for many other similar tasks. This does imply a particularly powerful church if everyone does things the hard way just to keep with church doctrine, but a powerful church is accurate to medieval time period.
Mages are specialists
You could say that magic is so hard to learn that a mage needs to specialize on very specific and exact set of abilities and spells to do anything useful. As such a hand full of 'standard' mage specialties will exist, being good at the few spells that are most needed and highly payed, with no one learning how to do other spells that aren't worth specializing in. More importantly a mage can not generalize his specialized magic to do much outside of it's specific intended use case. Once you have that then you only need to look at the hand full of common mage specialties and work to make sure none of them are game breaking.
For instance maybe a mage specializes in sensing the wind and (slightly) controlling the winds who will work on a large boat helping to propel the boat to it's destination. Say he doesn't create wind so much as redirect it in the direction he wants (conservation of energy!) and has decent but not exact control. He could be quit useful on a boat, helping the boat get to where it needs to go much faster and avoid storms, but he can't create winds so powerful as to push people around, or so focused as to move items at a distance, and he still can only work with existing winds. Effectively he is really only useful at assisting wind powered tech like sail boats. Now you need to ask what it does to your world to have boats travel faster and somewhat more reliably (and this is a non-trivial change to your world! still lots of world building comes from it), but you don't have to put much though into other affects of magic because this is really the only thing wind mages can do well enough to be worth doing.
This does offer an interesting story premise of having a mage who does generalize specialized skills to unusual use cases. The creative 'guile hero' who lacks strength but uses what limited abilities he has to maximum effectiveness is a standard and successful trope someone may wish to try writing. Though if one does go that route remember anything a single intelligent person can come up with on the fly has probably been thought of by someone in society in the past, so be careful not to come up with any innovations that are too effective or people will start wondering why it hasn't become standard yet.
Mages are rare
One common trope is to make people able to work magic rare enough that they don't significantly change the world. Perhaps a mage could move some supplies form point A to point B faster then someone can carry them with a cart. However, if you have few mages and their all busy maintaining your key palace spells and doing other more 'glorious' work it may be that you still have people using horses and carts most of the time. You can make your culture mostly fit simply by limiting the availability of alternatives.
This is a double edged sword. It keeps your peasant life standard, but since your nobles and king are rich their have a mage or two on staff, so your have to put some thought on how it furthers the divide between peasant and rich and how the rich employ their mages, but it at least limits the scope of your changes somewhat.
If you take this trope far enough, with mages sufficiently powerful and rare, you get into the question of what happens when one powerful mage goes rogue and tries to use his power to force others to do what he wants; since he is so much stronger and there are not mages to counter him. Of course one easy counter to this is a knife to the stomach, even massive power is not very helpful when anyone can kill you before you cast a spell if they get close and your not 100% vigilant. In fact this is likely the honest answer to 90% of rogue mage problems, that a solitary mage, for all that he can cause devastation when focused, can never be strong enough to protect himself from reciprocation if he starts abusing his powers too blatantly, someone can always knife him in his sleep unless he has a major power base/country backing him and offering protection. Thus most mages, even if massively powerful, will either work within the law or stick to subtler bullying and manipulation because if they draw the anger of the king/country they will not be able to stay vigilant enough to defend against an entire country focused on prosecuting them. You can also use counter-magic tricks as mentioned earlier to address rogue-mage problems.
Limit magic Power
You already mentioned this, keep power limited so that magic is not too massively powerful. However, power does not just mean raw strength, ie how much does it break conservation of energy. Power also means control. You can limit effectiveness of mages this way as well.
Using magic telekinesis to pinch an artery to assassinate someone may not work because no one has that level of finesse. Perhaps a mage could produce a fireball hot enough to roast a half dozen men, but it's going to take him so long to do it that he's likely going to take an arrow to the gut (or knee?) long before he pulls it off, making it impractical to use. Maybe fine magic require extensive focus, so that if you want to prevent someone from doing something like your needle trick you just need to distract them, they may be able to use less subtle magics easier but fine magics are hard to do without the support of everyone around you.
Combining a few of the above options together, the "net magic" of your world needs to be limited. That means the total ability to modify things using magic should be only so great. You could spread lots of power near a few people, or a little power near almost everyone, or some combination with lots of week mages and a few breathtakingly powerful ones. However, in the end the total available magic a society has to work with should have an upper bound.
