As a follow up from riding flying creatures to attack castles, let's assume that the practice is now so prevalent that a standard castle is no longer defensible.
How would the fundamental design of a medieval castle differ if it was designed downwards instead of upwards in order to counter fliers?
I'm trying to figure out what major changes might be made. I imagine that wastage, lighting, stabling and defense would all be impacted heavily. I suspect they would need to avoid flooding somehow.
Underground fortifications are very defensible, far more so than any above-ground fortification. However, the main reason that they are rare is the time and expense in excavating them, amongst other reasons.
Assuming a medieval environment with flying mounts in common military use, underground fortifications make a lot of sense, and their added defensive value compensates for the additional expense and the other problems they have.
Most of us should be familiar with medieval surface castles, and there are plenty of internet resources describing them for those that don't, so I won't describe them here for the most part.
The first difference between underground and above-ground fortifications is their attackable area. A surface fort has a large attackable area, effectively the whole perimeter in most cases (including their upper aspect if there are flying mounts), whereas an underground fort has a very small attackable area, effectively only any openings in the ground.
Like a surface fort, an underground fort would have traditional architecture facilitating an active defense such as arrow slits, murder holes, portcullises and gates defending its openings to the surface. Gates and portcullises, rather than being wooden and hinged, could be stone and round, so that they would roll into place sideways. Proper design would prevent their simply being pushed inwards, and a gate could be made immensely thick, yet easy to roll from inside with the use of a lever, making battering a suicidal tactic. A gate can be locked by the simple expedient of placing a wedge behind it. Arrow slits would have the additional advantage that there should be no question of them being open to the sky, making plunging fire as a means of attacking an unmachiolated wall (a machiolated wall has wooden hoardings that make this tactic less effective) of a surface fort ineffective.
Given the lower attackable area of an underground fort, it could easily occupy a very large volume of a hilltop, and yet require a wartime garrison smaller than that required for a similar sized surface fort in peacetime.
Now, we get to problems. Underground forts have the same non-combat problems as any fort, i.e. supply of food and water, with the additional complication of requiring a supply of air. While this can be achieved by simply cutting narrow tunnels to the surface, these ventilation ducts become potential avenues of attack in their own right. Since the fort is underground (that is its whole point), it is difficult - and should ultimately not be attempted - to deny an enemy access to the whole surface above the fort.
In some environments, such as dense woods, it may be possible to have concealed ventilation, and I would expect that many underground forts would have such. However, since an attacker could easily clear any such concealment by the expedient of arson, I would expect that there would be one, or more likely two, fortified ventilation duct headers. These would be basically a fortified chamber at the head of the open ventilation duct, accessible by a separate personnel tunnel. They would likely be designed to prevent the defenders of the vent header from egressing onto the surface for the simple reason that if the defenders cannot get out, attackers would have a far harder time getting in. The effect of a defended vent header would be that they could deny an enemy the ability to smoke the defenders out for as long as the header remains manned with active defenders.
Since the issue of ventilation would be so important in an underground fortification, it would not surprise me if a Sodium hydroxide carbon dioxide scrubber was invented in the medieval period. It may not be known what such an item does to maintain air quality, just that it does, and Sodium hydroxide can have a secondary use as a particularly nasty substance to drop from a murder hole onto an attacking enemy. It also requires hydrated lime, which is easily made/regenerated by cooking limestone in a kiln to drive off the CO2, and hydrating it with water. A Carbon dioxide scrubber would greatly increase the amount of time the defenders of an underground fort could survive an attempt to smoke them out, by the simple expedient of closing the vents and activating the scrubber. This would mean that the limiting factor to survival would not be CO2 buildup, but oxygen supply, which typically lasts longer than the time for lethal levels of CO2 to build up in a confined space. With only a small number of defenders and a large internal volume, survival could be extended potentially to weeks in a completely sealed situation. Smoke scrubbers in the form of water-filled tubs through which outside air could be bubbled could allow reduced ventilation in a smoke-attack situation for quite some time, and water would also absorb a number of other potentially toxic substances that could be used in an attack. Water would naturally be on hand for purposes of drinking, even if there was a well (which is likely).
Another problem that an underground fort has is external visibility. In a surface fort, defense is achieved through high structures, which provide the bonus of being a good vantage point over the surrounding terrain. An underground fort would lack this natural advantage to some degree (depending on where it was built), but in situations where there was limited height advantage over the surrounding terrain, a single tower could be built, rather like a minaret, which would be relatively easily defended from ground and airborne attackers, and could be abandoned easily if attacked (by which point the defenders should be aware that they are under attack).
Since the arguments for location of an underground fort are similar to those for a surface fort (a high geographical feature), flooding is unlikely to be an issue, however drainage of waste products would be an issue, requiring somewhat better than historical medieval plumbing, but not impossible to achieve, since the Romans who predated the medieval period had already accomplished the necessary engineering.
Derinkuyu, Kaymaklı and Özkonak underground cities in central Turkey, built around the 8th-7th century BC, could be considered early precursors to medieval structures such as I have described above.
Building underground fortifications against flying enemies faces 3 major issues (in addition to the usual issues when a castle was besieged):
Bombardment. If facing flying enemies conducting a siege, the carrying capacity and altitude of the attackers determines how large of a rock can be dropped on your fort and from how high. Whatever is exposed will need to be able to withstand this bombardment, but that is not necessarily significant with proper construction.
Tunneling from above. Miners breaching your walls is bad enough for a castle, but underground you can also have sappers attacking from right on top of you. You still need to maintain control of the land above and adjacent to your fort.
Ventilation. Buildup of harmful gasses will be your enemy - maintaining good ventilation will be incredibly important. This is not hard to accomplish with ventilation shafts, but protecting those ventilation shafts may be very difficult if enemy sappers can be air-dropped on top of it.
Construction of a fortification which could withstand aerial assault would be incredibly expensive - these would be more regional citadels than small outlying posts. The best means I can think of would be to essentially create an artificial hill surrounded by a moat.
Clear a wide area somewhere stable (on rock would be preferable to minimize sinking, but having a spring or well inside would be a lifesaver), and begin by setting very thick internal walls (at least 5 feet of stone depending on the size of the hill you want, maybe go Roman and use concrete) to create corridors and rooms with arched ceilings. These ceilings can take copious amounts of dirt piled on top to harmlessly absorb the impact of dropped stones.
You would need three layers of exists - sewers, entry gates (air intakes), and chimney.
Sewer drainage is one of the reasons you wouldn't want to dig down, as you probably want some outflow of effluent (unless this is only inhabited during a siege, and even then throwing the effluent on the outside of your hill might be unpalatable for the occupants). You also don't want to go down or you might get lots of water leeching in through the walls. Outflows would need to be protected, but just multiple outflows too small to crawl through would probably suffice.
Your 'ground level' will need to have fortified gates, with plenty of arrow loops around the edges to fire at attackers, and these also operate at air intakes for your ventilation (if they try setting up fires to smoke you out, just close the doors to those rooms so there is no longer a draw through the gates or arrow loops). These vertical walls will be very resistant to rocks dropped from above - you are still vulnerable to trebuchet, but they might not ever be invented with flyers dropping rocks available, and smaller siege engines could be vulnerable to defender ballista.
On top of the walls, you have a soft mound of dirt which would easily absorb the impact of dropped rocks. At the top of this hill you would have at least three 'peaks'. These are walls with arrow loops to shoot any anyone who tries to land troops on the hill, as well as the cave-like openings to vent the central chimney. You need multiple so peaks so archers in each can defend the other from enemies landing and digging from above. This is still a vulnerable area and not much can be done about that - covering them in iron spikes and thorny bushes would certainly help, but over the months of a siege they can be bombarded and burnt.
The dirt cap protects you from dropped rocks, a large moat prevents outside tunneling, and archers from miners landing on the hill and digging from above. The pressure differential between the lower openings and the upper chimney should create a steady draw through your fort, providing plenty of ventilation.
If you want an added bonus, try to breed a small flock of birds (or miniature dragons depending on your world) which are highly territorial against other flying creatures. If they nest in the thorny brambles or in birdhouses in the sides of the peaks, and do a mass 'defend against the predator' flock attack against any large man-carrying creature which comes close to the hill, attackers would be restricted to very high altitude bombings which would be so inaccurate that their dropped rock might not even hit the hill at all (good luck precision bombing an arrow loop from the side).
Alternately, a rookery in the peaks to house their own intercepting rocs/dragons/pegasi/whatever would help there (and probably be popular from an offensive projection of force or scouting as well).
Living in the fort would be a bit dark, the only natural light coming through the arrow loops, so most of it will need to be lit with oil lamps or rushlight. I would imagine most living would be done out in the surrounding lands - even during a siege, a rooftop garden might be high enough to escape the slings and arrows of the ground troops while providing plenty of sunlight, with people retreating inside the giant bunker during an air raid.
In a way i think it depends on how the fortification came into being. Was it designed to be a fortress-town by an architect under the orders of a rich person? In which case most areas will have a similar style with a strategic or militaristic layout, the fortifications in this likely carved into the rock itself.
Or did it build up naturally as township -> which may have started with a single underground path through the mountains that was part of a trade route, which then built up as people dug out dwellings along the route which then grew into a town, with fortifications later built for further defence.
Also (and i know this has been somewhat covered by other replies), think of the basic needs of the people living inside -> Air, Water, Food, Waste Management. Now, Air and Waste Management have been covered a bit by other people, but where do they get their food & water? If built into a mountain, then streams of rainwater or meltwater will likely be the source - maybe with controllable aqueducts if the water is only available for a part of the year, a nearby river from the valley could be another source (but would be hard to bring up to the fortress, and might be a bit inaccessible when raided... unless there is an underground bit which can be accessed by a well...) Food is another problem, do they get food from outside? in which case what do they trade for it? Is the land above and around the mountain farmed? in which case how do they keep large flying creatures from ransacking their land?
furthermore, on the subject of trade, where is the mountain located? is it near rivers where boats can bring materials from all around the land to be worked and resold or near the sea where fish and shells can be gathered and traded for other goods? Is it a strategic location along a trade route and therefore gains much of its food from taxing? - All of these can affect the architecture and style of the fortress
and okay, maybe this doesn't help answer your query... i kind of went on a tangent, and maybe focused more on the habitable aspect in a town/village kind of way then a fortress, but i hope this helps
One issue that is important with underground architectural designs is that to allow access, in tunnel sited fortifications, (which was what I kind of assumed you're asking about) defenses can't be continuous. Where a castle can simply have a major wall in a direction where attacks are likely, thus forcing attackers onto less favourable approaches. In order to have an accessible fortress in an underground tunnel you have to have gates and thoroughfares right through your stronghold.
