Yes and No.
This line:
"tech = 1920-ish, but no airplanes. Maybe blimps."
Makes me say 'No'.
A lot of answerers have quite rightly pointed to pre-WW2 Germany and how they trained an airforce in secret - however, they have omitted one key point - the reason Germany was able to do this was due to the extensive German gliding Clubs that existed from the 1920s onwards.
This ensured that there was an extensive knowledgebase of aviation understanding and piloting skill.
In the 1920s, heavier than air flight was already well established and understood and so a Gliding program wouldn't (and didn't) raise any eyebrows.
So - if heavier-than-air flight is unknown in your world building scenario - we have a pretty big hurdle to get over. We can't have a cadre of trained/skilled pilots. Unlike Germany, we don't have a population base with all of the transferrable skills needed to pilot a powered aircraft.
And the problem here is that the skills required to pilot an Aircraft (or simply navigate in 3D space) are pretty unique. If we look at other secret projects in history (say the various exploits of the Skunkworks) - there's a significant amount of overlap from ordinary aviation - meaning that the development of the niche/specialist skills represents a small fraction of the overall skills needed - The SR71 had the challenge of getting your head around the speeds being travelled at (you've never been lost until you've been lost at Mach3) - but that's merely an enhancement to the existing skill of navigating an Aircraft.
The development of Heavier-Than-Air flight is such a groundbreaking achievement that I doubt it would be possible to be kept secret.
However...
1920s Ships - the amount of AA Firepower they wielded could be summed up as 'Non-existent to Pitiful':
Ma Deuce didn't really exist in the 1920s - and wasn't widely adopted until the 1930s.
Oerlikons didn't exist in the 1920s.
Dual-Purpose 5-inch/38-caliber gun didn't exist in the 1920s
The Bofors gun didn't exist in the 1920s.
And perhaps most importantly of all...
Proximity fused Projectiles didn't exist in the 1920s either.
Now - to be fair - all of the above (except the proximity fused munitions) came into being in the mid-1930s. What you see a lot of in WW2 is that various ships that initially had little to no Anti-Aircraft armament got retrofitted in the late 1930s or after the likes of Pearl Harbour to get a significant upgrade to their anti-air capability.
Also - to be fair - the threat level of 1920s era aircraft is likewise miniscule - they are almost exclusively biplanes (insert Fairy Swordfish British Meme) with very limited bombing capacity...
But we have to take a pause here and talk about the 1921 American Airpower test - where TL;DR - a number of ships were attacked by Aircraft in a trial - the initial payloads of WW1 size bombs (100Kg, 600 Pounds sizes) proved to be ineffective - but Billy Mitchell (absolute legend) arranged for 1000 and 2000 pound bombs to be dropped - and with 3 near hits under the waterline to a captured German Battleship - they were able to sink said Battleship (although this test was highly controversial for a number of reasons - the Navy asserting that a well-trained Damage Control party would have prevented the sinking).
However - we must consider that the above test took place after the advancements in air-power that were driven by WW1.
I will add a little aside here - In WW2 - most of the anti-ship dive bombers ended up using old Battleship calibre Armour Piercing Shells (12 inch, 14 inch etc.) with some modifications to turn them into bombs. Whilst this was done in WW2, so 1940s tech, the actual munitions used were older variants of Battleship shell - Essentially all those pre-Treaty (The washington Naval treaty - I think 1926?) Calibre shells. Since the shells exist in the 1920s - I'm allowing it.
To Conclude
From a pure 1920s technology perspective - most of any naval Superpowers ships are going to have very limited Anti-Aircraft firepower. It is possible, with 1920s Technology to successfully sink a Capital ship. That said, most of this innovation came about from the results of WW1.
Furthermore - the most effective Anti-Ship weapon (Modified Battleship AP shells converted to Bombs) sort-of existed.
However - without a means of training pilots or at least having a segment of the population with enough transferable skills to be able to do this all in secret is a big stretch.
My suggestion:
Have gliders exist in your story, but make powered flight unknown - this allows you to solve the problem of having Pilots. You could also introduce a sport whereby Glider pilots practice dropping things from their cockpit, to see who can get it closest to a target.
Now you've got the skills, you've got aerodynamic understanding - your countries key advantage is making an Aero Engine (light and powerful enough) - you could still keep this secret as you could have the planes cut the engines when coming into land.