Things in the WWI era were much easier to hide, since the means of collecting and collating large amounts of data did not exist the way the did in WWII or today.
Even if the secret base was dug by contractors, it could take months for the contractors to get back home via ship and train. Shipments to the base could come from Argentina, the UK or America, leaving massive gaps in any observer logs. And communications over long ranges was very low bandwidth. One famous example from WWI was the detachment of the armoured cruiser SMS Emden from the German Pacific fleet. The single vessel was able to cruise the Indian ocean, tie up almost 100 allied warships searching for it and terrorize the sea lanes taking prizes for a period of three months and steaming thousands of miles. She actually spent some time in places like Dutch Batavia where the locals had not yet been informed of the outbreak of WWI, and treated the ship's crew as visitors and guests.
So long as the lab's staff is clever and patient, they might not even have to carry out a lot of overt security measures. Even the cover story could be kept in that era by simply doing things in stages: hire one crew to clear space for a resort, hire another crew later to dig the foundations for a mine, hire yet another crew to build a hospital for the natives, etc. Using tramp steamers on an irregular basis makes things even more convoluted, you break the pattern of regular shipments, and captains are not going to look too closely at manifests if the price is right.
So the interesting part of the story would then be how the intelligence service hunts these clues down and pieces them together. The clues will be all there, but widely distributed across both time and space, minimizing the damage that is caused by people talking, and keeping things compartmentalized so even inadvertent clues such as the ideas dropped in other answers don't add up very easily.