Rubberized cloth
Preferably was available/known to humans in our world up to the year
1920 (preferred, but not obligatory, especially in cases of a durable
alternative which has all other 3 properties).
Charles Goodyear invented a process for mass producing rubberized cloth in 1844. This material involved impregnating natural fiber cloth under tension with vulcanized rubber resulting in a material that was highly flexible but not too stretchy, strong, fairly light weight, resistant to both very hot and cold weather, impermeable to air, and very waterproof. All good properties for bat like wings.
Being durable enough not to require the membrane to be replaced often
(by not needing to be replaced often, I mean that, ideally, the
membrane could withstand a reasonable amount of damage before using it
for flight becomes highly risky).
This is where woven natural fibers alone like cotton, silk, and wool really fail. Their edges need to be properly finished or the whole material comes unwoven under very little stress; so, while something like silk might be able to check most of the boxes, a small hole or burn is all it will take to make the whole wing fall apart under the stress of flight. More textured fabrics unravel less easily, but they are also more combustible. Either way, they can not check all of your boxes.
Rubberized cloth overcomes this problem. Like concrete reinforced with rebar, the materials lend what each does well to the other making for something much stronger than either material on its own. The cloth becomes the basic structure of the fabric giving it it's strength while the rubber binds the fibers in a way that prevents them from unwinding or unraveling. So, even if rubberized cloth does get a hole in it, the hole will not just split like purely woven textiles do. This means even very old and somewhat damaged wings will still maintain most of their structure without you having to worry about a cascading failure in the material.
If you need to make the wings even stronger, you can add reinforcing wire to the weave. This is basically how modern tires are made and it's all based on this same process invented by Charles Goodyear.
Hard to catch on fire.
Depending on what exact properties you want the wings to have will determine what fabric you wish to use. If you were to use asbestos as your base material, it will be a bit heavier and weaker than some fabrics, but VERY heat resistant since you are basically looking at what fireman's coats are made out of. Keep in mind that such wings could be used to fly or go through flames unharmed but not at the same time. Asbestos unravels very easily compared to other fibers; so, if you heat soften the rubber too much, the asbestos will not be able to hold the wing together under its own tensile strength when flying. But it can survive the heat well enough that the rubber can melt staying in place, then re-cure after they cool back down.
That said, even without asbestos, vulcanized rubber does not burn very easily at all. Direct exposure to an open flame might melt a hole in your wing, but they will not combust. This is because the rubber will pool at the edges of the hole smothering any flames that might try to burn the inner cloth and making a reinforced edge so that the hole does not easily become a weak point in the structure. So, unless you plan on running into burning buildings with these wing, I would just go with cotton, linen, or silk as the inner material since these will be stronger, lighter, and more flexible than asbestos. Silk would probably be best, but also the most cost prohibitive.
As waterproof as possible.
I think it goes without saying that you won't find an answer that beats rubberized cloth on the water resistance issue. The rubber fills any gaps in the cloth's weave preventing the wings from holding onto or being penetrated by water at all. So, your robot would never need to wait a moment for the wings to dry: one good flap would cast off any droplets that might be clinging to the outside and you are good to go. I'd even say, the wings themselves would make for an ideal raincoat for your robot if the body itself maybe does not do so well with water.
But how strong was it really?
Finding spec sheets that date back to the 1800s is kinda hard, but we know that they would have probably used vulcanized latex over industrial manufactured cotton or linen. So, the specs of that should be very similar this: https://therubbercompany.com/rubber-sheeting/commercial-rubber-sheeting/economy-natural-rubber-sheeting. This material has a tensile strength of 30 kg/cm2. So, even at only 1 millimeter thick, it would still be much stronger than your peak stress expectation.
This spec sheet also tells us that it can operate to spec at temperature ranges of -20°C to 70°C which will cover you for anywhere ranging from well below freezing to much hotter than any desert. That said, many natural fibers can reach temperature ranges in the -150°C to 100°C range without significantly weakening; so, even if the latex weakens, the underlying structure should still hold at much more extreme temperatures as long as the wing is not so damaged that you need to worry about unraveling.