You don't need magic to explain pre-industrial airships, just more abundant and accessible sources of naturally occurring helium than real Earth has.
Hot air balloons are (arguably) pre-industrial; certainly they require only Renaissance-level technology to manufacture. Your limitation there is the fuel; to get a big enough hot-air balloon to consider it an airship, you need a fast-burning fuel which means having a patroleum industry, or at least an advanced coal industry. Thermal Airships also have a number of drawbacks.
However, helium airships remain desireable, and are limited mainly due to current shortages of helium. But if your pseudo-Earth had multiple natural gas pockets with large reserves of helium, and some of these were close enough to the surface to be reachable via pre-industrial mining techniques, then you could have a source of helium for airships.
There are some handwavy aspects to this (like, an Earth with more helium would also have more radioactive elements near the surface, which would have numerous other effects), but IHMO it beats inventing a whole system of magic.
This also gives you some story hooks, because control of the helium mines would be a source of wealth and politically contentious -- likely even a cause of wars.
There's three other aspects to the airship. One is a frame, which is the easiest part. Laminated woods were quite advanced by the late 17th century, and work quite well for light but strong frames, so much that the largest airplane in the mid-20th century was built that way. So, likely fantastically expensive because of the amount of work involved, but possible.
The envelope is more of a challenge. Helium leaks fast, and we're talking cloth here. Rubberized cloth existed late pre-industrial, but would be quite heavy. I don't know enough other materials science to say whether other cloths available in the 18th century (like silk) would have been able to retain helium sufficiently well. Although again, losing altitude can be a story hook. (per Sebastien below, Goldbeaters Skin would have worked, although would have required slaughtering huge herds of oxen)
The final bit is propulsion, which would most likely be fan-shaped "oars" driven by humans, unless you want to advance the invention of the rotary fan by quite a bit. This would effectively make your airships "floating triremes", and they wouldn't be able to go very fast, or take on any kind of headwind to speak of.
That's the "no magic" solution. If you want "minimal magic", you'd be applying it in 3 places: the extraction of helium, the gasproofing of the envelope, and propulsion (like making those fan-oars more powerful than they really are).