TL;DR: It all comes down to who can make the framework - and who has helium.
Blimps are fairly easy to make - at least, low-quality blimps are easy to make. In theory, all you have to do is insert some hydrogen (or helium, if it's cheap and plentiful) and let the thing fly. Sure, you have to make major modifications so you can actually steer the thing, but you can get it off the ground easily. The same goes for hot air balloons.
Rigid airships are a wee bit trickier. You need a very strong internal structure that can resists extreme pressure and keep the lifting gas inside. You can use small "cells" of gas
Image from Wikipedia user 84 user, currently in the public domain.
as shown here, but you still need a large framework - as shown here, the skeleton of the USS Shenandoah:
Image from Wikipedia user Saperaud~commonswiki, currently in the public domain.
You can make the lifting cells rather easily, but it's the rigid structure that presents the big problem.
The famous Zeppelins, often considered to be the greatest airships ever built, used a metal framework to hold everything together. Count Zeppelin's first creation, the LZ 1, used aluminum. It's more famous cousin, the LZ 129 Hindenburg, used a framework of duralumin, a strong aluminum alloy. Duralumin actually was first widely used in airships.
Duralumin was only created in the early 20th century, but aluminum as a material had been around for a while before that. It has been used in classical times, although it had really only become easy to make in the 19th century. It is surely easier to make aluminum than to make an aluminum alloy, and so it would be chosen first when developing an airship.
Summary and Conclusion
I would argue that the most important consideration to take into account when trying to determine when a civilization could build a rigid airship is when they are able to develop metal appropriate for building airships. Obviously, lifting gases are essential, but they don't forbid a civilization from making blimps and other non-rigid airships. A civilization probably couldn't create a rigid airship before it used lightweight metals, such as aluminum, or before it used metal alloys, such as duralumin.
Regarding Mark's point about the Schutte-Lanz airships: Indeed, rigid airships could be made of wood. However, it wasn't exactly the best choice for large ships because moisture tended to compromise structural integrity. As Wikipedia says,
Wood composites had a theoretical superiority as the structural material in airships up to a certain size. After that, the superiority of aluminum (and later duralumin) in tension was more important than the superiority of wood in compression. Schütte-Lanz airships until 1918 were composed of wood and plywood glued together. Moisture tended to degrade the integrity of the glued joints. Schütte-Lanz airships became structurally unstable when water entered the airship's imperfectly waterproofed envelope. This tended to happen during wet weather operations, but also, more insidiously, in defective or damaged hangars. In the words of Führer der Luftschiffe Peter Strasser:
"Most of the Schütte-Lanz ships are not usable under combat conditions, especially those operated by the Navy, because their wooden construction cannot cope with the damp conditions inseparable from maritime service..."