While this is quite admirable (placing realistic limits to magic), the real issue comes from finding the source of the energy to do so. Human beings are not particularly energetic in the larger scale of things, so unless magic involves making tiny and subtle changes, manipulation of physical objects, changing temperatures and the like will almost certainly result in the death of the magic user as they exceed the limits of their own bodies.
Tapping into external energy is also problematic, since you now need to have some sort of plausible means of doing so, not to mention a sort of channel to move the energy through, "insulation" so you don't get killed by overloads, and perhaps a magical equivalent of things like transformers, voltage regulators, surge arrestors and so on. At that point, I'll just be reaching for my trusty ACME device.....

Oh this will do nicely....
The alternative I might suggest isn't to actually go for direct physical manipulation, but explore the quantum multiverse. There is some finite (even if very tiny) probability that something you want is happening or exists, so a careful magician could be carefully tuning in to the multiverse and "seeking" the timeline where a particular rabbit is in a particular hat....
This is possibly as fraught with problems as physical manipulation of the universe, so beware.

Wait a sec, what is the probability of "that" happening?