Using only what is available in the 18h century, across any culture, what might airships look like in such a culture, assuming that an inventor used the resources he had to create one.
You must have aluminium -- there is no substitute for its combination of strength and lightness; and to smelt aluminium you need copious amount of electric power, which was not available in the 18th century.
The Hall–Héroult process for smelting aluminium was invented in 1886; before that aluminium was more expensive than gold and silver. As an anecdote, emperor Napoleon III of France "is reputed to have held a banquet where the most honored guests were given aluminium utensils, while the others made do with gold", or at least so says Wikipedia.
You must have internal combustion engines -- there is no substitute for their high specific power. Steam engines are too heavy; they need large high-pressure boilers, they must carry the water with them in addition to the engine and the fuel, and open flames don't mix well with hydrogen (or they mix rather too well, depeding on the point of view).
The first primitive internal combustion engines were invented right at the end of the 18th century, with practical engines appearing towards the middle of the 19th. The Diesel engine (which was actually used on actual airships) was invented in 1892.
Some comments suggest that wood may be used to make the structure of airships. There are after all historical examples of airships with wooden structures. The first problem is that, as the linked Wikipedia article says, "the airships became structurally unstable when water entered the airship's imperfectly waterproofed envelope, [...] during wet weather operations". The second problem is that [above a certain size] "the superiority of aluminum (and later duralumin) in tension was more important than the superiority of wood in compression". It's the same as with airplanes: small airplanes can be made of wood, larger airplanes cannot; for airships this is even more important, because in order to be actually useful airships must be very large.
(I have converted the comment into an answer as advised by user Shokhet).
The first demonstration of a prototype balloon was in 1709 (Father Bartolomeu de Gusmão).
The world's first air force was founded 1793 - French Aerostatic Corps
Option 1: Take Father Francesco Lana de Terzi's vacuum design (1673), and permit money and time to be focused upon it.
Option 2: A variant of Lieutenant Jean Baptiste Marie Meusnier's balloon ship with bird-like attachments (1785).
Option 3: A glider type of ship (Swedenborg, 1716), taking advantage of the ability to dynamically fly (which would have to be learned).
Option 4: A combination balloon glider - that uses its ability to change altitudes to gain switch direction and engage in dynamic flying with fixed wings.
All of these are possible in the 18th century - they just needed funding and focus to make them happen.
Steam engines have problems of weight once you include the boiler - and 18th century materials technology REALLY wasn't up to the sort of pressures you need in a flash steam (or water tube) boiler.
However, there has been at least one pedal-powered airship!
Make that at least two ... the White Dwarf, built in 1984
piloted by Bryan Allen, famous for flying the Gossamer Albatross across the Channel in 1979. I believe White Dwarf still has the world record flight (58 miles) for pedal powered airships, and cruises at 6 or 7 knots.
and a French one, on which Stephane Rousson apparently failed to cross the Channel in 2008.
It's not totally beyond belief that one could be built using 18th century materials - very light woodwork or bamboo, silk, and hydrogen gas - though performance might be less than modern standards.
How might airships be made using 18th century technology?
I think the answer is: using 1920-1930 aerodynamics and flight mechanics knowledges, late 18th century steam engines knowledges, wood and fabric.
Remember Clement Ader's 1890 steam engine produced 20hp for 51kg, which is 2.5kg/hp, while Wright brothers 1903 internal combustion engine produced 12hp for 75kg, which is 6.2kg/hp.
An 18th century Watt's double action piston steam engine, made out of steel, could have produced enough power. Even if it was 20hp for 150kg (7.5kg/hp)
Put such an engine on (let's say) a 1930 Piper Cub, or a Fiesler Storch, and it will fly. A Piper Cub is mostly made of wood and fabric, which was doable in the 18th century.
Main problem was the lack of knowledges in aerodynamics, but if guys like Otto Lilienthal, Wright brothers and Gerhard Fieseler were born in 1750, and met an other steam engine genius, then first human heavier than air flight could have been achieved in late 18th century.
They might be able to build zeppelins, as they do not require a lot of technology to make. They could be made with layers of textile, which has been made airtight with beeswax. Or as dot_Sp0T commented, use goldbeater's skin to reduce the need for beeswax (I think it will still be needed to seal up the seams).
The propulsion could be done with wooden propellors, powered by manpower.
Moving down could be done with having a hole at the top that can be opened or closed, and moving up could be done by a system of a fire heating air inside the balloon.