Well, to give an answer that is going the complete opposite direction as the others...there is another option. Build out your world in such a way that 'Magic' is the core of how the world works.
A good example of this is some chunks of the lore around the tabletop RPG Exalted. In that world, things fall to the ground not because of the natural attraction between large masses, but because the beings who designed reality decided it should work that way, and the pattern spiders operating the Loom of Fate make sure it continues to be the case. If something happened to the Loom...or even just to the local Weave, "physics" would start to malfunction. A few examples they give of what might happen if the weave is messed up...doors may stop opening to the right location. A construction worker may fall to his death...and simultaneously be standing safely on the scaffolding, having watched himself plummet to the ground. And in extreme cases the color orange might start tasting like the number fish.
Another pretty good example is the Stormlight Archives and the way Spren act and interact with the world ('washing the hands wards off rotspren')
In this case...things that look like 'Physics' are actually just a manifestation of the natural magic of the world. Fire is not a rapid oxidation reaction, it is a magical phenomenon that consumes certain materials as reagents to maintain itself...but is difficult to control and will run rampant if given more of the reagent it can consume. The river doesn't flood because it rained too much, it flooded because the local river spirit is angry. A thrown object doesn't sail through the air because of Newtonian Physics, it does so because the magical forces that make up the world declare that it should move that way. Smelting iron isn't a matter of refining existing materials from its raw form by heating the ore to a point that the iron melts while the stone does not--it's a matter of a formalized ritual for fusing together Earth and Fire to create a strong material. Iron doesn't rust because of oxidation, it rusts because the bond between the Earth and Fire was imperfect and interaction with Water and Air can cause the bond to start to break down. Washing your hands isn't removing germs, it is a cleansing ritual to clear them of malignant magical forces.
A world like this can look just like our world, functionally. Smelting can look the same, fire can act the same...but the underlying mechanic could be magical in nature, rather than physical. And this could result in some things being very, very different from how they are in our world. For example, in Exalted...gunpowder doesn't exist and cannot be created, because the world doesn't actually operate with regard to the periodic table. Sulfur, Saltpeter, and Charcoal may exist, but they are all varying combinations of Elemental Earth with other things, and don't combine to create an explosive.
In such a world, Magic is simply making use of a deeper understanding with, and connection to, the way the world actually works. In some worlds it could be a science that can be researched, studied, and understood by anyone who put forth the effort. Alternately, the world may have special requirements to break away from the 'standard' rituals (like striking flint and steel together to call Fire)...again to bring the example of Exalted: the world functions atop a force/energy/matter called Essence, and unless you have the ability to channel it, you can't do anything 'supernatural.'