Assuming no one can cheat or think outside the box because the outside is somehow toxic or hostile to human activities, then military forces will devolve into Infantry forces, and the Infantry forces will have their characters based on the wants and needs of the owning polity. (more on that later)
Since urban operations are pretty intensive, the first thing that will become clear to every force is the need for lots of manpower. Attackers will find themselves rapidly reduced to small, squad sized forces operating in very tenuous contact with the main body due to the nature of urban geography (once you turn the corner into an ally or enter a building, you are out of direct contact with the larger formation). Defenders have the same problem, in order to "cover" approaches and have forces available to meet the attackers, they will also need lots of manpower.
The next issue will be what sorts of weaponry will be appropriate? Fighting will be done at short ranges for the most part, so the bulk of the soldiers will have weapons that work best at close range. Since buildings act as impromptu fortifications, there will be a need to breach walls and clear closed spaces, so weapons like grenades, flamethrowers, short range rockets with large warheads and so on will be developed. In a medieval setting, many troops will become miners, sappers or engineers to do the same things with picks and shovels, making an opening for a grenadier to throw the flaming hand grenade into the hole...More modern settings will see the development of submachine guns, assault rifles and carbines, shotguns and handheld grenade launchers, or their futuristic equivalents (Metalstorm, anyone?).
Vehicles, if used, will be heavily armoured to withstand attacks at close range, including IED's, mines, suicide bombers, RPG's or whatever exists in your scenario. They will fight back with weapons that can deliver high volumes of fire, or explosive breaching charges. Engineer vehicles will also be heavily favoured to clear paths and demolish strongpoints. IDF vehicles like the ACHZARIT and PUMA have evolved in this direction for this reason. Tanks, if they exist, would probably evolve into smaller fire support vehicles much like German StuG's of the second world war, capable of moving down narrow streets and having a short barrelled weapon firing high explosive shells rather than a long barrelled, high velocity cannon.
In all cases a very robust logistics organization is going to be needed to push supplies forward through contested ground and retrieve killed and injured troops and damaged equipment. Getting meals forward will be a huge chore when every corner could expose the logistics troops to sniper fire or an ambush. "Loggies" will also need to be adept fighting men, in addition to doing their usual duties on the front. (the deep rear troops like clerks and third line maintenance will probably be deeply resented "REMF's").
As for the character of the militaries, much will depend on the owning polity.
A polity which is mostly interested in protecting itself and not in conquering may develop a universal citizen militia on the Swiss model, where everyone has training and everyone carries and keeps military grade hardware at home or on their person. Invaders will not meet a conventional coordinated defense, but literally have to fight house to house as every single person will be capable of fighting and will be protecting their own personal property. A small permanent force might exist to use technical devices like AFV's, electronic warfare or protect from chemical and biological attack, and a skilled engineering force will exist to reinforce and rebuild structures both during an assault and after hostilities cease.
A polity which is interested in conquest will invest in legions of troops, well equipped and disciplined, capable of going into these deadly fights with as much manpower and equipment as necessary to overcome resistance. Only fairly large polities can afford the manpower and resources to do so, so only a few "empires" will exist in this setting.
Smaller polities which feel the need to project forces may evolve a more commando like force of raiders who can swoop down the tunnels and deliver a swift blow to knock their opponents back o their heels before the enemy can react. Once again, depending on the setting we can be talking anything from Ninja's to SoF forces. Given the restricted access through tunnels, they will mostly operate under cover to get in and out of targets. Soviet Era GRU "Spetsnaz" troops were often concealed as truck drivers for long range trucking firms, crewmen on merchant ships, communications troops in headquarters units (every HQ has lots of signallers) and embassy personnel, to give you an idea.
And of course every polity in this setting will have large and effective intelligence units busy placing spies and agents in every other city, in order to get up to date maps, profiles of leading citizens and military commanders, economic data and so on. The real war will be going on continually as intelligence forces battle counterintelligence forces in the shadows between actual hostilities.