If magic is real, can it be true that rational scientific thought should exclude it?

"Magic and science coexist." A tired phrase by now, because it's found in so much fiction. But can it make sense?

Assume a world broadly parallel to our own. It has experienced its Age of Enlightenment and Industrial Revolution.

There is also magic (type yet undefined). It should allow human magicians who cast spells, whole species that are magical in nature, locations and objects with persistent magical properties. The usual stuff.

The Masquerade is not in effect. It is, and always has been, impossible to keep a fundamental part of the nature of the world secret.

How to keep magic separate from science?

"Science" includes many fields of study. How to make it such that magic is not one of them? How to make magic resistant to study by experimentation?

Science must be "logical" and "rational" and magic not. A person with a rational mindset should not look to magic as the first answer. Why?

And how and why would the scientific method ever have developed if a major part of reality was resistant to it?

I left the nature of magic open because that's an inseparable part of the question. What type of magic could fit this world? Is there any magic system that can make sense here?

I've asked this question on other sites before. A couple important points resulting from that:

Magic can't be totally random, since it must be possible to control it to some extent, or there could be no magicians.

The suggestions I previously received that came closest to satisfying my requests hinged on irreproducible results. These permitted spellcasting, but ruled out the other aspects of magic.

• Here I'm trying to answer a related question worldbuilding.stackexchange.com/a/36475/8465 – enkryptor Feb 26 '16 at 8:27
• (For some reason I cannot add an answer, even though I have enough reputation.) I just wanted to drop two quick ideas that I haven't seen in the existing answers: [1] "The magic" is actively tampering with the memories and written records of people trying to understand the fundamentals. The average practitioner and casual observer will see the magic work and accept it. But any attempt to dig beneath the surface is automatically recognized and 'erased'. [2] Whenever someone starts to understand the magic in any technical detail, they become utterly unable to explain it to others. – mhelvens Mar 10 '16 at 18:15
• Not everything that exist can be explained by science right away. We have use our brain since the beginning of humanity, yet today science still struggle to map it out completely. The same as magic, it can be exist as much and as global as you want in your world, yet their science still need hundreds or thousands of years to detect some kind of special quantum entanglement between the object and the spell caster's brain that happen everytime a spell is casted. – Hariz Rizki Dec 5 '18 at 9:02

I think the mistake you're making here is trying to categorise magic as something which is profoundly "not science".

If we boil down "science" to simply be the processes that we use in order to further our understanding and manipulation of natural phenomena, such as trial and error, or rigorous analysis, then there is no reason these cannot be applied to magic.

In our own world, we typically try to use science to understand basic hows and whys. For example, how do two chemicals react when thrown together in a vial, and why do they do that? By doing so, we discover the laws which the natural world follows. These processes can also be applied to magic, except when the magic does not have rules. That would make it very dangerous indeed, and quite frankly, something you'd probably want to stay away from rather than study.

So, unless your magic system has zero rules, to some degree or other, it is possible to apply scientific reasoning to it.

What then, discerns magic and science? We could draw in Aurthur C. Clarke's third law here: Any sufficiently advanced technology is indistinguishable from magic.

By that, you can take "magic" to be anything science fails to understand. Therefore, if you wish to have hardcore science and magic co-incide, there should be some core aspect of your world's magic which cannot be explained by science. Using that logic, magic technically exists in the real world, and simply represents things which we understand to be "base rules" or causes, rather than effects.

If that is not appealing, you could create a magic system which encompasses sizable differences from our world. For example, if all humans in your world are capable of manipulating traditional elements, (fire, wind etc) you could consider that your magic system, even if it has been largely explained by science. Remember, your readers come from this world, where such things do not exist, and thus they seem magical to the readers, regardless of how they seem to your characters.

• I think you may be slightly misstating the premise of this type of fiction. I believe it goes more like "there is Science, and then there is Magic, which is totally not governed by the laws of Science known to us (the 21st century reader)." It's not that Magic doesn't have any laws or experimental results, only that those results don't follow the same rules of nature that we the readers are aware of. – AShelly Mar 30 '15 at 21:56
• "Any sufficiently analyzed magic is indistinguishable from science." – Compro01 Mar 31 '15 at 0:43
• Compulsory XKCD: xkcd.com/435. Magic is either wa~y over on the left or wa~y over on the right. – Pharap Mar 31 '15 at 1:26
• @TristanKlassen When you phrase it that way, what make you think people discover the same set of rules of nature today? Consider the way people thought about motion before Newton's laws and Galileo's experiments. I find there is a lot of agreement on the middle ground stuff, but on the fringes of science, the work done looks an awful lot like how we'd have to discover magic. In fact, I remember Quantum Mechanics being called "magic" in an era before it proved itself out well enough. Wave/particle duality is MAGIC. As is Einstein's relativity. – Cort Ammon Mar 31 '15 at 14:37
• I think the man that got it most right was Sir Terry Pratchet (RIP). When one character in his book tries to explain weightlessness in space (science) to the protagonist, fails and arm-waves "It's magic", the protagonist (a wizard) replies "What kind of magic?". The point is that "magic" if it exists would be studied in a structured way. – Aron Apr 1 '15 at 3:11

If magic has effects in the natural world, it will be studied by science.

This isn't really escapable, science can study anything with observable effects in the universe. If magic is a well known force, humans will attempt to understand it through scientific observation.

The best you can do is make magic so rare that few people believe in it or have any opportunity to study it. Then it would look much as the world does today.

Separation doesn't seem that important. Science is not a thing, force, or entity. It's the study of things, forces and entities. It's a method humans use to understand all the things which happen in our universe, to put something outside of science is to say it has no observable effects on this universe. Thus, if magic is real, it will be studied by science.

• +1 for "Science is not a thing, force, or entity. It's the study of things, forces and entities". Indeed, science is not only physics and chemistry. History, sociology, philosophy and even theology are sciences too. – vsz Mar 31 '15 at 6:07
• +1. I like what you say. But I think other people pointed out ways to make it resistant to experimentation and it seems false to me to say that the only way is to make it extremely rare. Making it faith-triggered or making the effect destroyed by experimentation seem to be the most thrown around. Then again, you just create a more indirect branch of science, but I think this might be sufficient for the question. – 3C273 Mar 31 '15 at 16:11
• @3C273 Even if scientists can only study first or second hand accounts of the phenomenon, they will. Magic can be made resistant to experimentation (which seems like a good defense against it!), but that certainly does not mean it won't be studied with science all the same. That's my entire point, science isn't guys in a lab with beakers and clipboards, it's simply the study of natural phenomena. – Samuel Mar 31 '15 at 16:17
• @Samuel True. I realised that as I wrote the comment and it's a very valid point. Hence the "more indirect" part of my comment. I just think that magic being relegated to softer science might be enough for the question at hand. – 3C273 Mar 31 '15 at 16:45
• They way I see it, magic is just a force, like magnetism or gravity. You just can't see it or manipulate it because you're not magical yourself. – ggb667 Feb 15 '17 at 19:54

There could be various cultural or other reasons for science not to study something that decidedly calls itself magic, such as:

1. The magic is practiced by some sort of priesthood, and examining it would anger the god(s) causing magic to cease or other bad things. This may or may not be true, but the idea is active in the priests/scientists heads.
2. The magic is practiced in another society which science does not know of, or the culture of the culture with science does not consider magic worth studying. These barriers can be geographic or cultural.
3. Magic can not be studied. Perhaps its fundamental to the magic working; it requires faith or simply "no observations" of its inner workings, else the magic ceases to exist/function.
4. There are no tools with which to study magic. (That is, the scientists cannot detect anything because magic is undetectable or immeasurable by the available tools.)

Scientists sometimes suffer from their own cultural bias, leading them to incorrect conclusions or simply poor science. Eliminating and adjusting for biases is part of a scientist's job. Everyone who does any science need to acknowledge biases. You can take advantage of this in your world.

Rules of Magic

You can have several types of magic work in such a setting. This is especially true if there are simply cultural or geographic barriers between the magic-users and scientists. This can also work if magic is a relatively new or hidden thing, so that scientists cannot experiment upon it.

Your rules of magic could also vary from a group or individual to the next. This results in magicians but no formal rules for everyone. This would stymie most scientific investigations, because you can always find a disproof of any hypothesis. Reproduction of results is important in verifying and establishing theories we use to describe the world. The study of magic would be reduced to "there are sometimes rules," which does not really count as good science.

• I think to be effective 1 has to be at least in some sense true, otherwise heretics would arise and still have magic. I think your answer is more directly helpful than mine. – John_H Mar 30 '15 at 21:44
• This is a great answer. Part of the question was "how to make magic resistant to experimentation" and solutions are given that are essentially social rather than technical. This nicely sidesteps the problems with the technical solutions. – Keith Mar 31 '15 at 0:05
• I like "there are sometimes rules" as science. Like Newton said, "For most actions, there is an approximately equal and opposite reaction." Or Einstein's claim that the laws of physics are basically the same in pretty much all inertial reference frames. – KSmarts Apr 1 '15 at 15:31
• It would be hard for a scientist in your world to have a bias against Magic any more than for a Botanist here to be biased against Electricity. It is real, it works in its place, even if I don't use it in my studies. The reason Science is opposed to Magic here is that Magic does not exist and does not work. – Oldcat Apr 1 '15 at 19:25
• "Magic can not be studied. Perhaps its fundamental to the magic working; it requires faith or simply "no observations" of its inner workings, else the magic ceases to exist/function." - scientists are researching thinks that change when observed for some time now. Faith should not be a problem if the magic exists. And even if there is 'no observation' rule then when is is triggered - is there a minimum distance from scientist or how many people got healed compared with placebo elixir (you can get the data from magical hospitals). – Maciej Piechotka Apr 4 '15 at 8:51

Magic? Surely you mean Thaumaturgic Engineering!

An important point: engineering is not science, and science is not engineering. Engineering may use science, and science may use engineering, but the two are not identical, and often even dissimilar in scope and purpose.

We were able to use many properties of reality long before we were able to get a near-complete mathematical description of the underlying principles. You can be sure the Wright brothers had not worked out in detail the Aerodynamic properties of their machines before taking off, although I think they did build a wind-tunnel. We knew how to make gunpowder (and made, and used a lot of it) long before we understood the chemistry of rapid oxidation.

In a similar way, magic could be employed productively and on a large scale without being completely, or even partially understood. This would be done in the same way technological progress was made before the advent of theoretical scientific advances: trial and error, filtering out what works from what doesn't. Encyclopedic and procedural knowledge, rather than theoretical and formulaic.

Not Another Physics Sub-Field!

Magic is generally distinguishable from physics thanks to a few characteristics. In popular lore, magic tend to rely on sympathetic principles or contagion principles.

• Sympathetic: Like affects like. The voodoo doll somehow captures your liking, and thus gains power over you. There is no other visible causal association, so a hidden Jungian psychopomp mechanism must be at work. In other words, the source of magic somehow reads our minds, and if the conditions of similarity are properly met, the effect is transfered across the imaginary link.

• Contagion: Hurk was great warrior. You kill Hurk, eat his heart, become greater warrior. Or the creation of holy relics by sheer physical proximity to a powerful source of magic or holiness.

As you can see, these are not exactly physical characteristics. The sword in king Hurk's grave should be atomically similar to other swords, but its psychic context, from a human perspective, imbues it with some of the strength of the warrior that used to wield it so well.

So the language of magic is usually based on one of these principles. It must have its own internal logic, otherwise magical effects would be random, and so every yawn susceptible to summon a horde of demons. In that sense, it would be vulnerable to scientific inquiry: the basic correlates of magical operation could be discovered, and refined to a degree. However, insofar as the efficacy of the two paths described is a function of a psychic correlate (entering an ecstatic state to achieve the symbol -> target mapping via symbolic contagion or similarity), it may not be easily reduced to repeatable mathematical equations, and in fact will likely be even less understood than human psychology is currently.

So how do I escape the grabbing clutches of Science?

Magic is sympathetic, empathetic, holistic and irreducible. The sympathetic aspect requires a certain kind of mind, drawn to symbols and metaphors, people with a rather tenuous grasp on 'normal' reality. The empathetic aspect requires a highly emotive personality, the opposite of the analytical mind, but is vital for successful symbol-target binding. Your best friend here is the holistic aspect. It requires the practitioner to $$\Large\textit{enter an ecstatic trance-like state}$$ that is almost wholly right-brained, where the details of what exactly they do are lost, and the hole endeavor is holistic, where the whole effect is lost if a single part is removed, and thus irreducible. Why does it matter that the naked dance must start clockwise under the full moon? Who can say. But it works no other way.

• Reading through the other answers, I was composing a similar view in my head - now I don't need to write it! :-) – Neil Slater Apr 1 '15 at 9:12
• Randall Garett's Lord Darcy has forensic sorcerers using the Laws of Sympathy and Contagion as the basis for solving magical murders. – Oldcat Apr 1 '15 at 19:27
• The thing is though, a magic system that is sympathetic, empathetic, holistic and irreducible can probably still be studied. It just means you need to add layers of abstraction. Sure the person in a trance like state can't observe what they are doing. But someone else can. Sure Hurk's sword is only different psychically, but if those psychic effects go on to affect the mundane world, those psychic powers do in fact get turned into physical powers. How the psychic powers interact can be studied. The process by which psychic turns into physical can likely be studied and used. – Shane Apr 2 '15 at 19:25
• I think that the psychic term is the keyword - if magic only manifests when a consciousness is manipulating it, and this psychic effect can't be directly observed, detected or measured - only its effects on the physical world can. This will make any scientific methodology inapplicable (in the same way that it's impossible to find a rigorous answer to cognitive questions such as "does your red look the same as mine"). Also note that this consciousness isn't necessarily one that is residing in a physical creature - e.g. it can be an object, a place or a disembodied "ghost". – G0BLiN Nov 30 '17 at 19:27

Why must science and magic be kept separate?

Make magic a fundamental force of nature, with its own rules and manner it can be manipulated. Scientists can study the power, They know how it is produced, have SI units to define it's power levels, and measure its uses. In the future machines that utilize Mana as a power source will be as common as our machines that utilize electricity.

In essence there should be no distinction between magic and science, no more then we consider electromagnetism to be some special ability separate from science. It's a force, it does lots of really cool things, and science has worked to make it do more. The only difference is that humans have a natural ability to shape magic without technology in your world, whereas humans can't do much to create or modify electricity or magnetism without technology.

If you want to keep some level of distinction between magic and science in your would, just claim that your present day scientists have not fully articulated or defined how magic works, just as those in the Age of Enlightenment couldn't have defined what caused lightning bolts to hit, or how to protect their houses from them. Our science has only defined parts of magic yet, and much is still unknown.

However, long before we known how photosynthesis worked we had plants and greenhouses. You don't have to have full understanding of the world to work within it; in fact, we will never have a full understanding of the world, just increasingly better approximations. It could be that scientists are still working to fully understand how magic works, but that doesn't stop magicians from using the magic as well as it's currently understood.

• There is a standard unit for measuring magic: the Thaum. It is the amount of magic needed to create one white pigeon, or three billiard balls. – KSmarts Mar 30 '15 at 21:33
• @KSmarts How many thaums for a black pigeon? – JAB Feb 1 '16 at 14:00

The answer is quite easy: There are instances where science can only make general answers because

• the result depends on too many factors

To experiment with something, we need to hold some factors constant while we are able to vary other factors and see their influence. If there are too many factors, it is very hard to make general conclusions. Social science and medicine have a severe reproducibility problem because humans are so diverse. There are substances ("unsafe" ones) which could kill one patient while having a neglible effect on other patients. We have age, sex, size, weight, health condition etc. and even if you do exactly the same procedure to patients, the results can never be exactly predicted. Thus medicine needed meta-analysis, a statistical tool to get an assessment out of studies with different results.

• the result depends on factors which cease to exist when we try to examine or replicate them

This may sound funny, but it isn't. It happens e.g. that a motorbike shows a phenomenon only when we drive it; when we stop to look at it, it disappears. Because we cannot disassemble the machine while we are driving we are stuck. Another example is tribology, the science of friction. The problem is that "normal" friction happens with dirty, individual substances. Your shoe, freshly filled with gravel, on a parquet floor has a very wide span of possible friction values. When I try to nail down a more precise value by cleaning it, I may get a better one, but it is not the "correct" gravel-shoe-on-parquet value anymore. In quantum physics we have the problem that when we try to make a certain physical value as precise as possible, another linked physical value will get as imprecise as possible.

• the result is too rare or too individual to get meaningful answers

Freak waves were considered a yarn until they could not be negated anymore. They are very rare and they likely kill the observers, so their existence was doubtful. Your magic could be very individualistic, so that one apprentice cannot for his life conjure a small amount of heat while another one burns without sweat through a steel door.

Even then science will get to some very, very general conclusions: Magic exists, the effects can be observed, but trying to find some general observations which are always true can be a royal pain in the ass.

I see some people have problems imagining it, so I will now give an example of magic which will be almost completely inaccessible to rational scientific observation.

On the world Paradoxis, magic and science exist together. The scientists and engineers are making observations and products which work the way we know: They are consistent and reliable.

But Paradoxis also knows magic: It seems that sentient beings can influence their surroundings by a strange force. Some people are quite adept at it while others struggle with it. Its influences are innumerable and well-observed and documented: Molecules change their form, their temperature, their consistency. People can move and levitate objects, start fires, cool things down, speed up or slow down processes. What is interesting is that people doing magic need to "synchronize" their attempted magic with the environment, the time and location, with themselves and observers. They get in a meditative state and "feel" what is the right procedure and what is wrong; the mental procedure itself changes every time an attempt at magic is made. Sometimes they feel magic is now impossible, sometimes it is possible to make several conjurations at once or strengthen the effect. But whatever the effect is, it is never exactly the same one.

The scientists have an impressive database of observed magic, but no one was able to get any consistency out of it, so most of them gave up.

• The thing is, if you can't do science on magic, there is a very good chance you won't be doing science at all. The core idea of science is that through study and experimentation you can discover how reality works. If you remove that truth, the impetus to do anything science-like is greatly reduced. In the world you are describing, the scientific method is clearly and obviously false. – Shane Mar 31 '15 at 22:19
• @Shane I think you have a misconception considering science, you must make a distinction between "reality" and "model of reality". If science could discover how "reality" works errors would be impossible. The idea is that through study and experimentation you can build a "mental model" how things could work. That this really works and that humans can build mathematical models agreeing with observations in astounding precision is itself a wonder and not given. There is no conflict between fields like physics with excellent models and (important !) fields like medicine with shoddy models. – Thorsten S. Mar 31 '15 at 22:41
• What I want to say is that methods working in one field could fail in other fields.So you can have good science and engineering and still magic which could not be penetrated by "normal" scientific methods. – Thorsten S. Mar 31 '15 at 22:47
• The reason why we can build mental models of reality is because we know, or at least we have reason to believe, that reality is consistent. What you're describing removes that truism. Nearly every scientific advance started with someone saying 'Huh. That's weird. Let me try to figure out how that works.' If there is magic it would be 'Huh. That's weird. Oh, well, magic. I'm going to go do something else.' Magnets? Magic. Electricity? Magic. Odd planetary orbits? Magic. The world feels flat but ships go over some sort of horizon? The world is flat, the horizon is magic; don't fall off the edge. – Shane Apr 1 '15 at 14:13
• @Shane Some of our observed "reality" seems to be consistent, it does not follow that all of it (or that other realities) must follow it. So it never was a truth, it was an assumption which works often pretty well. Magnets, Electricity and odd planetary orbits did budge to it, but art as concept never had. "If there is NO way to put a normal nail with normal jelly under normal conditions inside a wall, then there is no reason to believe that sth other (hammer ?) can do it ?". Science is nice, but not overarching. – Thorsten S. Apr 3 '15 at 18:41

All right. This is a very good question. And there are some very good answers. Smart, thoughtful answers.

But, given the way the question is stated, it feels like those answers are still thinking inside the box. In general, magic is treated as if it were nothing more than a branch of physics. @Samuel nails this reasoning very nicely:

If magic has effects in the natural world, it will be studied by science.

This isn't really escapable, science can study anything with observable effects in the universe. If magic is a well known force, humans will attempt to understand it through scientific observation.

And @dsollen describes the inevitable outcome:

Make magic a fundamental force of nature, with its own rules and manner it can be manipulated. Scientists can study the power, They know how it is produced, have SI units to define it's power levels, and measure its uses.

It's a pretty rich and fascinating way to consider magic. +1 to both of those answers. I think they are entirely correct.

Entirely correct... as long as magic is understood purely in terms of its physical manifestations.

But physics is not the only human model of reality. There are other modes of experience. Other ways to think about what "magic" might be.

Pardon the little thumbnail, but anything that would do van Gogh's Starry Night over the Rhône justice would be a massive download.

I'm not going to dare anybody to try to use physics to explain the impact of such a painting. Because that's just luring people to be silly to make a point of some kind.

Art and poetry are not about physics. Neither are love, tragedy, horror, or transcendence, or mystery; or more subtle concepts such as "liberty" or "ennui". But they are all a rich part of human existence. We live them, or at least we understand, to some extent, when we hear them told. That's really the point.

So why do we want to force magic into the mold of scientific reductionism?

Consider magic in fiction. Here's something from The Face in the Frost, by John Bellairs:

He looked absently around the cellar as he waited for the pitcher to fill, and suddenly his eye was caught by the fluttering of an old cloak hanging on a wooden peg. And in that instant Prospero got the odd notion that the cloak was not his, and might not be a cloak at all. He stared intently at it as the fluttering of the garment became more agitated. And then it turned to meet him. With empty flopping arms it floated across the cellar floor, swaying in a sickening nightmare rhythm. Prospero clenched his fist and felt his pulse beating in his palms; he fought the rising fear as the cloak flapped nearer, for with all his heart he did not want it close to him. As it closed the gap between them, all the spells against apparitions ran through his mind, but he had the queasy feeling that none of them would work. The thing was about six feet from him, its cold musty-cellar breath faintly brushing his face, when it simply stopped. The flapping arms dropped, and the gray cloak, or whatever it was, slumped into a ragged heap on the stone floor. Prospero stepped back nervously and stiffened as he felt a cold sensation. But when he looked down he laughed abruptly, since he had stepped into the spreading brown pool of ale that was now sloshing and frothing over the sides of the pitcher. He shut off the spigot and leaned, trembling, against the barrel, his forehead pressing the fragrant wet wood. When he looked again at the place on the floor where the cloak had fallen, he was not surprised to see that there was nothing lying on the rough candlelit stone. The peg where the cloak had first hung was not there either.

This is serious, consequential magic. But it's not physics; it even ends by withdrawing the physical reality that the entire terrifying scene purported to represent. Still, Bellairs leaves us no doubt about the serious peril and horror it represents.

And there's this, from E.R. Eddison's [The Worm Ouroboros]:

When that was done, yet more biting seemed the night air and yet more like the grave the stillness of the chamber. The King's hand shook as with an ague as he turned the pages of the mighty book. Gro's teeth chattered in his head. He gritted them together and waited. And now through every window came a light into the chamber as of skies paling to the dawn. Yet not wholly so; for never yet came dawn at midnight, nor from all four quarters of the sky at once, nor with such swift strides of increasing light, nor with a light so ghastly. The candle flames burned filmy as the glare waxed strong from without: an evil pallid light of bale and corruption, wherein the hands and faces of the King Gorice and his disciple showed death-pale, and their lips black as the dark skin of a grape where the bloom has been rubbed off from it. The King cried terribly, "The hour approacheth!"

This is a short excerpt from a description of classical Alchemical sorcery, rendered in Eddison's masterful Jacobean prose. Its impact, its significance, has nothing to do with physics. It's pure wordsmithing magic.

Consider the possibility of a world in which reality, occasionally, perhaps not very predictably, responds to the same kinds of impulses and influences that art and poetry exert on human consciousness.

Consider a world that bears that kind of wonder.

• For an example of magic-as-poetry: goodreads.com/series/43409-long-price-quartet – Sobrique Mar 31 '15 at 14:04
• problem is that in such a world magic would still be pretty straightforward to subject to scientific analysis. Complex and weird is not the same as impossible to measure. whether you're willing to lie to fit in with a group in certain situations is not a matter of simple physics but there is still a branch of science dedicated to how people react in different social situations. How a crowd of scared people moves is not the same as how a gas moves yet there is still a branch of science which allows sports venues to be designed to avoid deaths during riots. – Murphy Mar 31 '15 at 14:10
• Maybe not scared people, but mosh pits actually do have a lot in common with molecular motion: npr.org/2013/03/22/174962714/… – 16807 Mar 31 '15 at 17:16
• Actually, science in the real world studies exactly such nuanced "magic" that reacts to art and poetry - human emotions and state of mind. It's not only one field of study but several ranging from psychology to sociology to political science all the way to neurology and cognitive science. If we can apply science to study human experience, we can certainly apply it to study such subtle magic. We even have very robust mathematical tools for studying phenomena that are hard to predict - statistics. – slebetman Apr 1 '15 at 10:21
• I think all you do here is argue that the study of magic would not be physics, not that it wouldn't be science. As @slebetman points out, psychology and neurology can and do study the effects of art on the mind. I see no real reason that magic as you describe it could not be analyzed by another scientific discipline. – KSmarts Apr 1 '15 at 15:23

Magic as an Art

One possibility would be to approach this difference not as a fundamental difference but as a difference of practice.

In the real world, we tend to draw such a distinction between science and art. Art can be studied, and in many cases described by science. There is science behind the nature of sound, science behind appealing compositions of color, and even science that analyzes what comprises an appealing composition of music.

Despite all this, we do not tend to view the creation of art as a science so much of as a craft. A sculpture is created not by looking at the science describing its internal structure but rather by fusing years of practice and hard work mastering a craft with an inner artistic vision.

Magic could function in a similar way. Science can look at the effects of magic and describe how a sorcerer draws energy through the earth, but knowing how all of these things occur does not give one mastery over magic, any more than knowing the science of sound makes someone a maestro on the tuba. Magic can be described with science, but ultimately the practice of magic is learnt through decades of hard work and perseverance in shaping the mind and body of the magician to perform it.

For some people, knowing both the practice and the theory behind magic could be appealing, in the way that some pianists may enjoy knowing the theory behind why certain chords sound the way they do, but knowing the theory is by no means a requirement to being a skilled magician, particularly if spells can't be cast quite the same way every time. Casting a spell would be a combination of being able to see the flows of magic around the wielder, having the intuition to see what kinds of effects those particular energies could have, and having the skill to carry out that vision.

It's possible that someone skilled in the science and study of magic could construct devices that could be used to do magic, much as a scientist can mix paints or build a piano, but these items would be useless unless placed in the hands of a skilled magician. A grand unifying theory of magic that would enable a scientist to actually build a device to do magic may also be possible, but requiring such a high level of knowledge and technology as to remove it from the realm of possibility for the next few millennia.

• I like that you point out that you can work with things described by science without being a scientist. This is true not just for art, but most professions. However, this focuses on the human aspects of magic - magic as practice. Unfortunately(?), the stories I'm trying to explain and/or emulate tend to have magic as an aspect of the world, and that is probably the more important part for my purposes. – Tristan Klassen Mar 30 '15 at 23:46
• @TristanKlassen I think ckersch is onto something, here, though. Considering magic as art, it may not even be something one practices, but is an expression of a personality's Id (ego, whatever). How you describe an event is different than how I do, just as how I light candles (via magic) is different than yours. Magic could still be innate, something that doesn't require actual intelligence. Animals might invoke types of magic based on instincts, and yet others might have an inherited magical attribute as a function of evolution... – phyrfox Mar 31 '15 at 2:45

What if the universe always works your way... But only for you?

People tend to assume that rules define how everything works.

That is to say:

(Rules/Reality) -> defines -> (Magic/Emotions)

For example, because of the exact position of every atom inside my head, their energy, and the set of rules known as physics, I feel happy today.

Or:

(atom+position+energy+physics) -> defines -> (happy)

But this becomes problematic very quickly. What about philosophical things, like "free will" or "choice". Are we actually incapable of influencing the world around me? Do we have a set, inescapable fate?

What if it doesn't work this way?

(Magic/Emotions) -> defines -> (Reality/Rules)

For example:

Because everybody expects the sun will to rise tomorrow, it will.

(expectations) -> defines -> (sun rise tomorrow)

So a "magician" is capable of influencing small amounts of reality, by refusing to believe the world works how it should. This allows them to create incredibly paradoxical situations.

Since scientifically-minded people understand very well how the world should work, they become incapable of witnessing magic that they don't already expect to work.

From the scientists' point of view, the universe follows well known rules, and responds as it should. They are incapable of understanding Magic.

From the Magicians' point of view, the universe is pliable, and they can break any rules in any way they desire, so long as they manage to overthink/overpower the minds of those around them.

From the layperson's point of view, both sides would be correct, to some extent. The layperson doesn't understand enough of the world to disprove either side, they are capable of witnessing anything, since they don't think anything is impossible.

Obviously, these "laypeople" don't actually exist. People would have various degrees of belief, somewhere between the two sides. This is how our world works, although it is currently dominated by scientifically-minded people.

Is the universe defined by how we think it works? Or are we defined by how the universe works? These two conflicting schools of thought are the very basis of science and magic. Of course they can't get along!

• So the dominance of science over magic (or magic over science) is decided by a popularity contest of the people present to witness the event. That would explain why magicians have to be such talented showmen. I used something like this in one of my unpublished stories, made for fantastic tension whenever magic had to be performed in public. Great Thought! – Henry Taylor Apr 2 '15 at 13:44
• @HenryTaylor This is how I personally hope our world works. It makes nothing impossible, and allows every "magic" system to work within it. :) – Kent Apr 2 '15 at 16:00

I think pretty fundamentally, there's a misunderstanding here. Science isn't a set of beliefs or a faith. It's a process. It's sort of like the recipe for making a cake - you put in the ingredients, follow the directions, and out comes a cake.

Science is a method - you follow the process, and you get a conclusion. Sometimes the conclusion is flawed, but sometimes your cake doesn't rise.

Magic... well, that depends what magic is in your world. It's not inherently at odds with science though - scientific method can be applied to anything - that's why it's so amazing.

I would imagine that 'magic colleges' aren't so much about magic, as they are about understanding in detail the rules and mechanisms to do it reliably. Which sounds pretty scientific to me.

The only things really resistant to science in the 'real world' are also the things that don't work. It's extremely hard to exhaustively prove that e.g. homeopathy doesn't work. You can only say that for every scientific study that has been done, there have been no results that indicate that it does.

With magic 'working' that wouldn't be the case. If enough people 'used' it, you'd have a statistical sample that - even if magic were generally a bit unreliable, you would still be able to observe and measure the outcomes.

So I would suggest the only way you could do this is to apply the 'guild method'. Accept that scientific study will work on magic, but seek to actively subvert and conceal the truth. If anyone 'knows' the truth, get them to join the guild, swear a (magical?) oath of secrecy, or ... ensure they keep their silence. (magically? Or just kill them). Keep 'science' as part of the 'arcane arts', and baffle the normal populace with mysticsm.

I think @PipperChip's answer is getting at the same ideas as mine for the most part.

Science is just a method of looking at the world; there should be a scientific way of describing all events even if the scientific community does not currently accept the way things are. Which is completely likely and probably your best bet; that the effects of magic are thought to be explained by the scientific community as being something else and the explanations dealing with magic, even if you do have strong evidence and experiments that can be done, would destroy ones career to do such that no scientist would ever dream of doing them, or at least of publishing the results if they do not conform to the reigning orthodoxy. There are, unfortunately, actually quite a few examples from history where this was the case; where a theory or a person had such influence that until the death of the main backers of that theory no new theory could be advanced.

The other thing to look at is the way that some religions view the situation of duality, and even the existence of magic, such that the primary existence of God is not to be seen in the effects of the world that science studies but in the internal nature of things. See for example hermetic alchemy.

Which gets to yet another possibility, again going back to religions, that science is again studying effects where as magic could be effecting primary causes. In which case it wouldn't be that science couldn't study magic, it would just not be in a position to do so currently.

Of course, even in those cases it would seem to be possible and even needed to use something of the basis of the scientific method for the development of the art of actual magic. Consider the staying power of Aristotle's theories and that I am drawing these last two examples from religion; where powerful orthodoxies that last for generations are more the rule than the exception. It is entirely possible to structure real magic such that it could be controlled in such a way that an orthodoxy controls it, and if belief or purity or something of the sort is required to access it the control could be effective such that they think they understand it and no one else could study it, not even most heretics.

One very fundamental assumption in scientific inquiry is this: the rules stay consistent. This is what allows us to make testable predictions and hence run experiments to determine how well our theories match 'reality'. Without this ongoing consistency, the scientific method cannot function.

The key to distinguishing magic from science, then, is to make the rules variable. In fact, if you describe magic as the action of pure, unfettered intention, the ability to make choices outside the framework of determined rules, then magic is literally the ability to either bend or break the rules.

Of course, truly unbounded magic is not especially interesting, as, in the extreme, it essentially leads to omnipotence, at which point any meaningful story is impossible, for there is no more possibility for conflict. The tension, then, is between the raw potential of unhindered magic, as an expression of pure, unrestrained will and what limited actions us human magic-users can accomplish within the frameworks of our minds that strive to categorize and tame the world around us.

In a sense, the science that defines our predictable world is the very mindset that limits our access to magic. The art of magic and the challenge of the magician, then, is to harness will and bend the rules within this framework of reality, without getting swept away by the raw power and losing all trace of human identity in the process. This is the balance of the mage, whose grasp on reality is always tenuous at best, for when the world bends to your will, you run the risk of getting twisted up in it.

• Of course, truly unbounded magic is not especially interesting, as, in the extreme, it essentially leads to omnipotence, at which point any meaningful story is impossible, for there is no more possibility for conflict. Caramon Majere would beg to differ... – Mason Wheeler Mar 31 '15 at 19:14
• I've read some interesting stories with near-omnipotence, but none with true omnipotence.. If the world works entirely to a character's will, then there is no more distinction between the character and the world in which it lives. A truly omnipotent magic user ceases to be a user of magic at all, but rather becomes the magic itself. As a destination, it leaves very little to be said, though certainly the process of arriving at (or avoiding) that fate could make for an interesting story. – Dan Bryant Apr 1 '15 at 20:30

Escaping scientific method

To study something in a scientific way it's necessary to put it into a controlled environment, or at least (and in that case you need a lot of luck and Occam's razor) to have independent observers who can inspect it on a regular basis and take notes. What can go wrong?

one-way interaction

no manipulation ⇒ no controlled experiment

Actually many things in modern science are observable but not accessible for manipulation and/or can't be put into a controlled environment (e.g. in astrophysics or in social sciences). In such cases it's difficult to make conclusions about causality or decompose a problem. We still can get an idea of what happens if we find something similar but more accessible to the inquiry, but if the nature of magic is alien enough it won't be possible.

Examples. Things like Clarke's monoliths if you explicitly label them as magical or an xxx-dimensional lovecraftian creature that perceives our universe as still images on a thin film. Both can imbue people and things with supernatural properties.

Limitations. Magic is supposed to be something that characters are able to utilise. So at most the source of magic may remain enigmatic but the artefacts produced by it still are subject to scientific research.

subjectivity

no independent observers

Inability to separate subject from its observer may also help, but only if it cannot be circumvented by some kind of indirect manipulation/observation. Consider dreams as a real-life example. Granted they were studied by Freud, but his methods have been discredited since then and afaik nowadays there is no rigorous way to analyse dreams.

Examples:

• a world where psyche... magical substances or spiritual practices actually get useful by greatly enhancing interpersonal interactions but hinder rational thinking (does love count as magic?);

• 'kafkaesque', 'solipsistic' or 'totalitarian' worlds where evidence has a tendency to disappear behind your back.

Limitations. Again, it may be hard to study these kinds of magic directly but one can investigate its secondary effects.

incomprehensibility

no regularity

Suppose the magic is projected by some kind of non-human sentient beings. Studying human behaviour is difficult enough since humans claim to have free will which translates to "I can mess up your experimental results just for the sake of it" and comes from the ability to reflect on the fact they are being observed and on their own past behaviour. But imagine the said creatures being vastly more intelligent than humans and/or — like the WoD fairies — able to tap into the collective unconscious so they can always subvert our expectations.

Example. Gibson effectively did it in the Sprawl trilogy, even calling them 'loa'.

Limitations. The only obvious issue with this scenario is that it looks more like human being an instrument of magic, not vice versa.

Sources of inspiration

Humanities are a field of knowledge where interesting questions can be posed but attempts at scientifically rigorous answers often fail. So they may be (and often are) used as a model for how people in a fictional world perceive and study magic (not every study necessarily is scientific).

How to keep magic separate from science?

"Science" includes many fields of study. How to make it such that magic is not one of them? How to make magic resistant to study by experimentation?

Keep the "hidden" effects of magic measurable only by magic.

For example: Where does the energy for the magic come from? How is it conveyed from the source to the target? A magician could answer these questions, based on direct observation/perception. Yet, to any non-practitioner there is no evidence besides the end effect.

It will be a necessity of the practitioners to make formal studies of their craft. It's how they better themselves and share their knowledge. The very nature of a spell implies that a magical force has been understood enough to be captured by a repeatable ritual.

However, explaining this to anyone that isn't a practitioner doesn't amount to much. They can see something happening, but that have only the practitioner's word on how it happens.

Science must be "logical" and "rational" and magic not. A person with a rational mindset should not look to magic as the first answer. Why?

Human magic should be limited to a small number of uses, but presumed to be capable of more. Perhaps the magic can only be used to produce scientifically impossible physical effects: levitation, teleportation, spontaneous energy/heat/light, etc. Superstition would then have people believe magic can: bend your will, make you sick, cure you disease, etc.

Why? Because that's how people work in the real world. Superstitions abound in every part of Earth. Demons, bad spirits, witches, angels, gods, ancestral spirits, have all been credited in ways that a rational, educated mind would not.

The field of human magic should be full of disagreement. Perhaps they disagree on the origin of their magical powers, or methods. Maybe multiple methods exist to execute the same effect. The lack of consistency will make it even more difficult for scientific scrutiny. Imagine there are 10 known spells that do the same thing (let's say lift a boulder), and that it's truly random which spells and how many will work for a given mage. Then add to it that even mages casting the "same" spell will have slight variations based on personal style, and it's a quantification nightmare.

And how and why would the scientific method ever have developed if a major part of reality was resistant to it?

Science can still classify phenomena and experiment upon magic, but can't reproduce it. For instance, there's no reason science wouldn't try to figure out how a massive, wingless dragon can fly. But eventually, they're only going to be able to record observations. Science would have no way of explaining the hows and the whys, and wouldn't be able to develop applications for their knowledge.

I left the nature of magic open because that's an inseparable part of the question. What type of magic could fit this world? Is there any magic system that can make sense here?

I think a magic system that is based primarily on violating the physical sciences would work very well. Lift things up, pull things down, fly, transmute, teleport, areas of silence, casting light or darkness, invisibility, and pretty much anything a Bender can do in Avatar/Korra in regards to elemental manipulation. The magic should generally not be able to affect biological or psychological processes (at least not any more often than necessary to seemingly validate superstitions). This precludes increased longevity, healing, cursing, mind control, etc.

The "strength" of magic should be, to the outside observer, entirely random. Some purely fictional quality or qualities should be responsible for magical ability. "Spirit", if you will. There should be no correlation between race, body type, size, physical strength/speed, general appearance, intelligence, personality, family/genetics, etc. and the strength of Spirit.

You can take this further, and have the number of mages also be random, not dependent on population size or geographic location. There could be no correlation between the number of mages from one generation to the next.

Even creatures that exhibit magical abilities can have randomness applied. Let's use the wingless dragons as an example again. Their max speed, maneuverability, acceleration, max flight height, etc. should have no correlation with their size, weight, number of spikes, color, age, etc.

There are any other number of stipulations you could put into place, that would make magic less appealing than science for explanations. A remote tribe could have a very unusual power, but they can only demonstrate that power in their tiny geographical area. There could be some areas/creatures that actively interfere with some technologies (or even electricity), that make it harder or impossible to take measurements. Certain elements or objects could be immune to magic for no apparent or explainable reason.

In the end, science would need only be aware that magic exists, is evidently controlled in some way by certain animals and humans, and that it can do things science can't explain. But, by its nature, magic would be unpredictable when trying to achieve consistent results. Relying on magic may work in the short term if the resources are available, but it'd be up to science to make the real achievements.

Personally, I think such magic would also spur science, as it would try to "compete" with magic.

So, your village has a guy that can start fires by snapping his fingers? Well, our village invented matches and now all of us can start fires whenever we want.

So, this 100-pound man can lift a 5,000 lb boulder with a wave of his hand? Well, this machine can lift 20,000 lbs with the push of a button, and we can build as many machines as we want.

So, you've figured out a way to train a certain dragon breed for riding? Well, we invented the airplane and we don't need animals to fly.

And so on.

A lot of people are saying that you shouldn't try to make magic and science separate, that it's not possible, so long as there are visible effects. But there's actually a simple way to keep them separate. Connect magic to something that we already know can't be studied: individual consciousness.

There are two ways to do this that come to mind.

Approach 1: Humans have no magical abilities, but magic comes from spirits/gods/capital-G God. The being(s) that control magic are either a) unpredictable and capricious, or b) go out of their way to avoid being studied and reduced to a formula. Either way, humans can only invoke magic by calling on these being(s), and, even though the effects may be observable, scientific study would not be able to find meaningful patterns in them.

Approach 2: Magic is completely individualized. One's personality and experiences determine one's magical abilities, but in a deep, complex way that isn't based on any observable patterns in brain chemistry. Even though a particular individual's magic may be analyzable, if that person's self-image changes in some drastic way, their powers may change accordingly.

Both of these would make for an interesting fantasy setting. Approach 1 would make magic look a lot like religion, and some scientists would likely scoff at it--even if the effects were plainly observable--because they wouldn't appreciate outside forces meddling in an otherwise orderly world. Approach 2 would probably result in scientists being desperate to understand magic, which could even be a plot point in itself.

Either way, although magic could, in theory, do anything, its effects would likely be random enough, and perhaps rare enough, that it wouldn't be the first explanation someone jumps to when they see some new technology or phenomenon.

• Your assumption is invalid. Consciousness can be and has been studied. We don't yet understand it but that doesn't mean that we will never understand it. – Tim B Jun 24 '15 at 11:20
• "Can't be studied" was probably the wrong way to word it... but consciousness, and individual behavior, could only ever be fully understood if it were 100% deterministic (no free will). And, even if one believes in complete determinism in the real world, there's no reason to necessarily apply that to a fictional world. – Adam R. Nelson Jun 24 '15 at 15:01

Arcanum: Of Steamworks and Magick Obscura -Troika Games

In the land of Arcanum, technology and magic unwillingly co-exist. High level mages are not allowed on the steam train because it might blow up. High level techies cannot safely disapparate and most spells will fail on them.

Tool use or spells will critically fail when done by a novice. Great mages fail at setting mouse traps and scientists fail the casting of the simplest spells. The higher the skill level involved, the greater the consequences are when the two meet. In close proximity of its 'opposite', failure is all but assured; epic battles would destroy everything or their spells would just fizzle and their circuits be burnt. -He says he doesn't like you...

You can play the middle ground but none of your work will ever be of much notice. Scientifically trying to study magic would get you killed. Using magic to understand technology would not work (also not advisable).

The best literature never reveals how magic works. If it does, it immediately removes my suspension of disbelief, as I know enough about the real world that that ain't right.

Raistlin Majere, on the differences of clerics and mages: (and the best description I've ever heard of how magic works)

She's channeling the power of a god, you dolt. I'm wresting arcane energies from the very fabric of the universe - it's completely different.

• aside from the land of Arcanum. Remember Nicolaus Copernicus he proposed the heliocentric model of the universe despite the fact that it is still inaccurate however my point is... based on his observation and predicting the position of celestial object and wondering the rotation of shift between day and night I must say that's science. As you see astronomy as we know it didn't exist then, as for magic time will tell... – user6760 Mar 31 '15 at 2:24
• @user6760 Agreed (I think), we do have magic; it's called calculus. Copernicus didn't have that. – Mazura Mar 31 '15 at 2:37
• that's make us smarter than him 100 folds... now that's magic [blushing now because I failed my precalculus test thrice] – user6760 Mar 31 '15 at 2:44
• The best literature never reveals how magic works. If it does, it immediately removes my suspension of disbelief, as I know enough about the real world that that ain't right. That's one school of thought on the subject. There's another, very different one that operates on its own distinct set of rules, and manages to produce some very good literature, perhaps best embodied in the writings of Brandon Sanderson. See Sanderson's First Law for an explanation of the differences between the two styles, and how to make both work well. – Mason Wheeler Mar 31 '15 at 18:30

And how and why would the scientific method ever have developed if a major part of reality was resistant to it?

You don't need to make Magic resistant to scientific study, just make it really complex. The scientific method did develop in our world, starting from a point when everything was magical. Why do we have astrophysics today, when it was once clear that the sun was carried across the sky in a chariot of the gods every day? It's because some things were predictable and rational, and we discovered we could not only make theories about them, but test those predictions.

If most things in your world are rational, then people will make a science of studying them, even if they find exceptions to the rules. Limits in the theories is what leaves room for "Magic". Back to astrophysics: We observe galaxies being pushed apart. We add up all the forces we understand, and find we need one more to make the math work. So we just say "Dark Energy". That's just another name for magic that we can observe, measure, but not really understand.

Science must be "logical" and "rational" and magic not. A person with a rational mindset should not look to magic as the first answer. Why?

Because it is hard and not needed most of the time. Why don't you think of special relativity whenever you see an object in motion? Because Newtonian physics has enough predictive power and much simpler math.

And perhaps because Magic doesn't play nicely with the rest of Science. Think Quantum Mechanics. The rest of Science deals with concrete objects with defined position, mass, velocity. QM has superposition of states, matter waves, and collapse of probability wavefunctions, stuff that just doesn't work like the majority of things you encounter everyday. Scientists know of it, and they know it can be described by complex mathematical equations, but it's just not relevant or needed to do their work.

So how to explain

human magicians who cast spells, whole species that are magical in nature, locations and objects with persistent magical properties...?

Just as we have super-tasters and people who are not red-green color blind, magic users and creatures may be able to perceive the fringes of the forces of magic. They can see the Quantum-Magical probability waves, and pull on them. But it takes lots of practice. And the math of it hard. Super hard. Only a select few wizards can really solve the equations needed to develop new spells - to predict that this incantation will have that effect. It takes extreme control to channel the forces of Magic to "enchant" an artifact so that it displays QM behavior on the visible scale.

Most people will use science-based technology over magic simply because it is easer to use and understand. Just like in our world. If I was gifted with great hand-eye coordination, and I practiced really really hard, I might also be able to do magic like Penn & Teller. But it's just much easier to walk across the room than to teleport. Compare this to the way crossbows replaced bows, and early firearms replaced both, not because they were better but because they were easier to use, and needed less skill to get an acceptable result.

I can think of 2 methods to make a magic system highly resistant to scientific inquiry.

Option 1:

Base it on the whims of a conscious, highly intelligent, almost omnipotent trickster god who can see the intents of all involved and takes personal issue with anyone attempting to subject his gifts to systematic/rational analysis and actively thwarts their efforts so that the closest anyone can get to serious understanding is that as soon as anyone starts even thinking about trying to understand it in the "wrong" way everything either stops working entirely or the magic gets malicious and turns on anyone trying to measure, quantify or analyse it.

Option 2:

Not totally immune to analysis, everyone is trying and it's useful but hard to predict.

This is the softer option. Make it anti-inductive like the stock market. However magic works it's linked to the ways that humans use it and the ways that humans think about it. A practitioner is free to subject it to scientific analysis and may even make significant gains, their power waxes as they plumb the secrets of magic and their apprentices gain power from the same secrets.... and then it all starts to not work. The flows of power adjust to the new "market pressures" as many people attempt to tap the flows of power and suddenly the rules and even meta rules have changed dramatically and unpredictably. In the old days it was possible to work out the rules that would remain stable for generations, years or months at a time but since the world has gone through a kind of magical industrial revolution things are just too fast as millions of people try to draw the available threads of power from the multidimensional knots they both make it more complex and drive faster change while the minds and dreams of humans themselves help create more and more complex mechanics.

I'm going to focus on the aspect that you want rational personal people to conclude that magic isn't the reason for things.

There is actually a really easy way to do this: make magic rare or subtle.

For example, suppose you are a rational person at a casino, and you watch a guy wave a wand and then win a single number bet at the roulette table. You are going to assume there is no magic involved, because it's not very unlikely a person will win a single number bet, but it is very unlikely that a wand waving person is actually a wizard rather than a fraud or a showman or someone just having fun.

And when the person wins ten times in a row, you're still going to assume there is no magic involved because sophisticated crooks and promotional events are far more prevalent in your world than wizards are.

Similarly, there are more circus acts going around with horses with horns glued on them than there are actual unicorns in the world. There are more tourist traps than truly magical sites. There are more clever toys you can make out of magnets than you can with magic.

Even if magic is common, you can still get this effect if your world has large numbers of frauds.

In such an environment, when faced with magic, rational people will not take it at face value, simply because it's so unlikely things are what they seem. Naturally, genuine rational inquiry will probably uncover the truth rather quickly depending on the particulars. Although your purposes may be served by the fact that there are plenty of "rational" people who are really just mimicking what they believe rational people believe, and they will continue to deny magic, even in the face of sufficient evidence it's really involved.

By far, the most straightforward solution would is to make your magic un-science-able. Since science is simply the study of natural phenomenon, you must make magic impossible to study, or people will do so.

Conveniently, we have a very, very loose analog for such a thing in the real world- quantum particles. Despite the fact that it runs counter-intuitive to everything we observe and understand on a day-to-day basis, simply observing a particle changes that particle's state.

Option A: Apply that same formula- magic exists, and its effects are predictable, such that a wizard knows his fireball spell will create a fireball and it will head in the desired direction. However, if you attempt to study or directly observe any individual aspect of the casting, the spell fails.

This, being a fundamental aspect of the universe, cannot be circumvented by logic- no amount of distance, abstraction, or ruse can "trick" the magic. It somehow, despite all logic suggesting such a thing to be impossible, knows that it is being studied, and therefore will not work.

Science will study magic to the furthest extent possible, reaching the scientific conclusion that magic cannot be directly studied or observed. Science will then move on to studying the effects of magic, IE, how much damage does a fireball cause, rather than the magic itself- much as the study of responding to tornado damage is separate, though connected to, the study of tornadoes.

Option B: To steal a page from the SCP Foundation wiki, magic is un-studyable for memetic reasons. Any observation you make of how magic functions, you'll forget. Any notes you write down, you won't be able to read. Any footage you take, you won't understand- or you'll forgot what you saw as soon as you look away. The only aspect of magic that scientists are able to remember and retain is this memetic principle- the world can remember that it's impossible to study magic, but it forgets anything deeper.

• "no amount of distance, abstraction, or ruse can "trick" the magic." General:"They're about to fire the etherium orb at us! we're all doomed!" Scientist:"There is a way! Johnson, prepare to make notes on the nature of the etherium orb spell!" – Murphy Mar 31 '15 at 16:12

It should allow human magicians who cast spells, whole species that are magical in nature, locations and objects with persistent magical properties.

As others have noted, this is impossible to do - anything this conspicuous would be examined by, and become, science. However, it could be that magic is more subtle - it can achieve a lot, but it's always explicable by some normal explanation, and even for the practitioners it's more a matter of faith that the ritual or w/e they performed resulted in the outcome. . . like a lottery win or a road accident etc. Science would therefore always find a more prosaic explanation, and find nothing to study.

• Maybe so. it should be noted that science can't always "explain" it's findings either; with things like drug tests or psychology or sociology studies, sometimes the correlation between two things is found before the explanation. Of course, many people are wary of results they can't explain, but my point is that as long as this magic affects the probabilities of achieving some desired results, it will be noticeable through statistical analysis. And of course, scientists would be very interested in being able to produce such effects. – zeta Mar 31 '15 at 6:35
• Ah, no, it has to be undetectable even by statistical methods. It would hide within the bell curve, in the space between the odds against winning the lottery and the fact that someone is going to win it. Therefore it would have to be more abstract than simply 'if you do x then y happens', and a run of successive lottery wins would be impossible to achieve. This is why even for the practitioners it's almost a matter of faith, because the correlation is so tenuous. – peterG Mar 31 '15 at 15:40

How they can be both not science and predictable

Something just can exist and cannot be studied. Consider if our world is in something like a black hole, we can still study things outside because we know the only ways things can be formed naturally. But if they are all sent deliberately by some intelligent species, and if they want, we will probably completely lose ideas, and still may found some of them useful. Otherwise they can state explicitly what they are, but that is still not science.

So if people know how to use magic just because they are told by someone (god, inner spirit, ancient unique artifact, coexisting species who didn't develop science yet and aren't cooperative, or even a talented scientist who calls something "magic" themselves and refuses to tell the details, etc), and those who tells the magic can change the rules of magic and even lie about them, and they decided that scientific ways are bad, then it would not be science.

Intelligence is one way to make simpler things very unlikely to be better. There may be other ways.

The same applies if people can request someone doing so, but have no way to force them to do physically. Or we can just allow people to build things where people can never know its inner structure physically again, and then forget those technologies to build them. If people just did build some weird mechanisms into it, it can be less likely to be studied well than natural things.

How they cannot be studied

After people have some ways to know them at the beginning, the other part is easy. But it might be boring to explain them in stories.

There are uncomputable things. Just make the magic mechanism behave like some of them. If casting a spell is just like writing a story with ideas which are neither boring nor previously known (the above part can explain why this can affect the result), science can define it but probably can't help improving it too much (without making things boring or known). Or easier: the computation requires a real number, and those who can manipulate real numbers can lie and are separate from our world.

But if there are other things related to magic which don't involve these (such as these are only required when someone's casting a spell, but not when someone's storing/collecting the magic materials), and are using tools those aren't intelligent, those mechanisms can probably still be studied.

There are other possibilities including:

• It is just too complicated to study.
• There are some exotic rules like if they are studied, they will be gone, but everyone has their own magic so they won't affect each other.
• It is just already too obvious that it doesn't worth any study.
• People don't want to share their knowledge for some reasons.
• It is based on some records of infinite length and people cannot copy them. The organization controlling them refuses to study for some reasons.
• It just has some rules like "the shortest and lexicographically first unprovable proposition is always correct", and we will never know it without being told by someone accessible to some proving methods unavailable in our world.
• Different magics can never be used together. And magic can only work if it is now in a situation in the common sense. Any non-obvious things never become useful.

For simplicity

If our emotions (which also have magical powers which can not be produced artificially in the setting) are not only sometimes not easy to control, but also always deliberately lies to our rational thinking, and studying them physically always break them immediately, then there are not many things that science can do.

And those cannot be studied are likely always more intelligent or strong if you are using the above excuse.

What cannot be avoided

If everyone knows magic plays important roles to life, there must be someone studying things related to magic. Say, the economics related to the existence of magic. I don't see a way to prevent that. If there is a way, it is probably making the science in this setting too weird. (Nobody cares whether they are considered science or not, though.)

People can still identify them in science. Such as the mass and energy conservation only holds if not involved some magic. Or it always holds, but all the magical materials contain unknown amount of energy (and no relativity here).

People can still believe how they are correct like gamblers.

If you want magic that can't be properly studied and analyzed like science, but also can't be so completely chaotic that it's not able to produce reproducible results (wizardry, etc,) then have you considered an invisible Flying Spaghetti Monster?

Magic is real, and is reasonably consistent, because that's the way that The Being That Takes Care Of Magic (TBTTCOM for brevity) wants it to be. But every once in a while, some scholar gets a little too big for his britches, trying to not only probe into what magic does, but why. And TBTTCOM doesn't like that, so TBTTCOM subtly shifts and twists things around until they either just sorta fail, or blow up in the scholar's face. Sometimes literally. Or sometimes even worse than that; did you ever wonder where some of the creepier mythical curses come from? It's happened enough times throughout the ages that scientific experimentation into the fundamental principles of magic has become the stuff of cautionary tales.

Which, of course, doesn't stop people from occasionally trying. Generally they come in two varieties: naive, idealistic and inexperienced young wizards who don't know any better, or great and powerful archmages who think they do know better, who have studied the mistakes of the past and know how to avoid making them... not that that matters to TBTTCOM; they'll just make some new mistake this time instead!

This question has attracted many replies, but few answers. I’ll make a stab at the latter.

Let’s be clear on the question:

Given:

• Something that roughly resembles typical genre-fantasy magic
• An Earth-like world and history, no earlier than the late nineteenth century
• Institutional science, recognizable to us as such
• Science does not study magic

Implicit:

• Science does not study magic because it doesn’t succeed

• Science does not succeed because something about magic is resistant to scientific investigation

• Scientists do not as a rule find this resistance interesting enough to investigate on its own

Questions:

1. How can this be possible?
2. How would the magic work?

To summarize, the majority of answers are of the form, “If it’s real, it can be studied by science.” These answers are non-answers, as they disregard the givens of the question.

Proposal:

Thorsten S. made an excellent point, having to do with the number and nature of factors or variables. In essence, if there are too many factors, and the factors themselves vanish or alter as soon as they are perceived, experimentation is likely to fail. This got a lot of nasty answers, but Thorsten was quite right.

First, we might simply suppose that a magical phenomenon is a dual-ended chaotic system. As Edward Lorenz put it, “the present determines the future, but the approximate present does not approximately determine the future.” So let’s define our spell as follows:

     P1    →     F1
P2    →     F2
Spell       mana        Effect


In other words, there is a determinate connection between the spell and the obscure, postulated power that will in turn generate the actual effect, again deterministically.

Unfortunately, each spell’s performance depends on a range of variables, and when very small changes are made they produce wildly variable results. Since each magician must to a great extent find his own way of manipulating mana, this means that two magicians can produce very much the same results without their methods apparently overlapping much at all.

Second, we can invert the problem by lodging magic beyond the outer limit of scientific knowledge. I don’t mean that it’s unknowable, only that, as several distinguished anthropologists have argued, magic postulates unlimited determinism. The classic example comes from Evans-Pritchard’s study of Zande witchcraft and sorcery.

To simplify, here we have Uncle Herman, sitting having a snooze beside the old granary. A high wind comes up, the granary falls over, and Herman dies. Explain.

• Scientist: wind, dry-rot, etc.
• Magician: wind, dry-rot, etc. – and witchcraft

What’s the difference? The scientist has not in fact “explained.” He has assumed that, in the absence of definite evidence to the contrary, there is no determinate connection between the forces that caused the granary to fall and the fact that Uncle Herman happened to be sleeping where he was. But that is an assumption, based on a complex system of logical investigation, accepted practice, and so forth.

The magician, however, refuses to accept this. He lives in a purely clockwork universe, as it were. It is simply not possible that there is no determinate cause of Herman’s death. “Random” means nothing—it’s a dodge to avoid having to explain anything.

Result

If we accept that both aspects of the proposal are valid, then we have a field in which scientific investigation is essentially pointless.

On the one hand, experimentation doesn’t work well, because you can’t isolate the variables. To investigate fire spells, for instance, you can either study how Magical Mandy does them, which is at least consistent, or else study how lots of magicians do them, which has statistical value. Unfortunately, the first study is statistically meaningless, because you can’t compare to any other cases, while the second produces completely inconsistent results (two magicians use paint, the other 98 don’t; two different magicians yell words that have Z’s in them, the other 98 don’t; and so forth).

On the other hand, these magicians seem to infer and manipulate causal mechanisms that simply aren’t there. The diviner, let’s say, can with 100% accuracy tell you what’s on the back of a card or whatever, and double-blind tests confirm this. So what do you learn? Nothing, because he says it’s obvious that this card is the Queen of Hearts because of the pattern he saw in his breakfast cereal this morning, and so far as anyone can tell, he’s telling the truth as he sees it. To him, the world is tied up with all kinds of weird forces, and he can “prove” their reality through this kind of divination, but they don’t respond to any other kinds of tests. They’re just not there. So when you do big statistical studies, the odd magician will stick out like a sore thumb, because normal probability just doesn’t work the same for him.

Ultimately, nobody in the science business is going to waste time and money on this nonsense. It’s real, sure, but it doesn’t mean anything. There’s nothing to be learned from it. You can’t build anything with it. If you involve real magicians in any kind of testing, stuff goes haywire, causing damage and expense. So let’s just ignore them.

Yes, in theory you could study these effects, but what for? What sort of nut is going to waste his life trying to figure out why Magical Mandy sucks cough-drops after casting spells? And for sure, nobody’s going to fund his experiments.

Q.E.D.

Well, I can see an easy way to keep the two alive.

Magic sucks. It allows individuals to do great feats, but it's not really worth the cost. E.g. it takes way too much study, it only applies to a random bunch of individuals (say, it depends on some genetic marker, or maybe there's two species living on the same planet - one has it, the other doesn't).

Scientific engineering, on the other hand, is available to everyone with the push of a button. Only the initial investment is costly, once you know how, you can make thousands of the machines and quickly outperform any wizard. Even if there's wizards with more power than any machine, they cannot be duplicated - building the same machine 1000x just takes some amount of work; having 1000 wizards means 1000 lifetimes sacrificed to study and practice.

Basically, wizards are crazy. They sacrifice an absurd amount of work just for a measure of individual power - but there's no way they'll be able to use that power to e.g. rule the world - while there's no single person who can challenge the power, there's no way they'll be able to fight the whole kingdom. The reason most people don't even bother studying magic is that it simply isn't worth it. Way too much work, way too little gain. It becomes a lot less attractive when anyone can learn it - Harry Potter feels exclusive because he's one of a fraction of humans who can ever practice magic; if it was just another skill that anyone can learn, like reasoning or math, it loses a lot of its "I AM THE MASTER OF THE UNIVERSE!" appeal. When you yearn for magic powers, you don't usually care about the power itself - rather, it's about having the power while others can't. Why isn't rationalist thinking considered a superpower in our universe?

Now, eventually, science will enable mass-production of magic artefacts and magical machines and whatever. We're not there yet, and we have no idea when that will happen. Maybe the source of magic is a machine on itself, and we just have to stumble upon it at random (detecting power coming from somewhere while observing wizards with sensitive instruments?). Maybe we'll be able to reproduce the genetic marker, or the complex machinery required to cast magic, and maybe we'll be able to make studying magic a lot less work, taking a lot less practice (e.g. the equivallent of steroids for magic-muscles). But we're not there yet. Until that happens, magic will have its place separate from the non-magical populace.

Just make it so that science hasn't yet figured it out. You're stuck otherwise; anything that's real is 100% compatible with the scientific rational process and there is no way around it.

But there is no reason why magic can't just be something that science has figured out yet. You could have lots of scientists struggling but failing to figure it out.

It would be the same as how rationally-minded people struggled with chemistry 1000 years ago – despite a) being quite able to do some chemistry, and b) lots of "magic level" chemistry happening within the human body, quite observable but totally incomprehensible at the time.

You could say that magic needs a certain state of mind to work. It's practitioners could have to rely very much on intuition. That state of mind could be incompatible with doing scientific investigation.

I had exactly this same problem for my setting, and the solution was twofold.

1. Split the world, having magic in one half of the world, and in the other half of the world, have science. The split in the world is in a fourth dimension, making physical travel from one side to the other difficult. The two philosophies are incompatible in their most advanced states.

2. Magic allows for effects that cannot be produced by science (yet?), but magic comes at a cost that limits its use and making it expensive.