There is this tabletop RPG that I love, called GURPS. It stands for Generic Universal Role Playing System. The creator tried - and IMO, succedded - in creating an RPG system which can model any setting and scenario, from fantasy with elves to cyberpunk to FTL science fiction.
The initial GURPS system in general, and its offsprings GURPS Fantasy and Gurps Magic in particular describe the way magic interacts with the world in a way similar to what you seek.
In these settings, mana is a term for both an energy source and a type of energy that can be accessed by living beings, and mages are people who are more sensitive to it. Every area in the universe has a level of mana, from absent to very high. When mana is absent, no one can cast magic. When it is low or normal, only mages can cast spells. High and very high levels of mana allow anyone to cast spells as long as they have the knowledge to do so.
The basic system stops there. GURPS Fantasy describes a world where some
creatures (such as dragons and gryphons) are mana-dependent - they will get sick and possibly die if taken to areas with low levels of mana. In areas where mana is high in concentration, animals suffer mutations that would make the movie Anihilation seem mild. Another aspect of its high mana areas is that they are random, and sometimes boost one type of magic while weakening others.
Very high concentrations of mana have two aspects, though, which can cause large scale "natural" disasters:
First, the amount of energy available for spells is, for all practical purposes, unlimited. Even the most unskilled mage is able to summon and control a category 5 hurricane, if they have the patience to keep channelling mana for a few hours.
When doing magic, skill checks are required. In GURPS dice rolls, critical errors happen less than 2% of the time, (compare with 5% of the time with systems such as Dungeons & Dragons). So consider that for every fifty spells cast a day, one will fail. Well... A failure in a very high mana area is a very spectacular thing. The caster may loose control of the spell, the spell may be cast in reverse, the caster may lose a limb, a hostile demon may be summoned... And the intensity of the mess will be proportional to the energy being chanelled.
So if you are channeling a Cat 5 hurricane and you trip, you may end up throwing a large chunk of the planet's atmosphere into space, messing up the whole planet's weather system for a few months.
In the story of GURPS Fantasy, elves once tried to perform a ritual in an area with a large concentration of mana to banish all orcs to another dimension. They failed, and as a result:
- A range of mountains turned into sand and formed the world's largest desert, destroying a dwarf kingdom in the process;
- The whole area occupied by the desert became devoid of mana;
- Instead of banishing the orcs, the spell brought an uncountable amount of fantastic and mythological creatures from other dimensions to their world, including every fantasy race and monster that is neither orc, elve or dwarf. That is how humans, centaurs, giants, goblins, gnomes etc. made it into their world;
- The effect above never stopped and has been going on for centuries.
Drawing inspiration from GURPS into your world: even the slightest act of magic in an area with a lot of mana may produce a doomsday scenario, so magic would be outright outlawd in such areas, and access to them would probably be controlled with an iron fist by that world's nations. Without access to magic, anyone on those areas would have to resort to technology instead.
On areas with no mana, magic is not possible, so reliance on technology is also common.
But in areas with moderate amounts of mana, where failing to perform a cantrip will not cause a volcano to sprout from beneath your house, magic is tolerated and even used daily. Since it makes things easier, there is little to no demand for technology in these areas.
Make mana concentrations vary through time for interesting effects and seasonal chains of migration.
Edit: as for how mana could interfere with physics more directly: mana could cause the ground to vibrate at frequencies humans cannot detect, for example. This could cause buildings to collapse. Or mana could make the air more viscous, so sailing would be completely different while still allowing animals and people to breath. Mana could change the thermal properties of ores.
If every area with high mana concentrations has a different effect, some things which quite easy in some places might become quite hard in others. Without anything to explain such differents other than magic, science would be a far more complicated thing in this world.