Dwarves don't have cavalry because cavalry is useless in their preferred environment.
Let's take a look at history.
Where does the cavalry excel?
On large, open flat plains.
Horses are large, heavy and require room to turn around. Footsoldiers can turn on the spot, rotating formation in place. Horses can't turn like that, thus they need a lot of area to manoeuvre. Because of that, despite what films will show you, cavalry is useless in melee - horses will be quickly wounded and riders dismounted, completely eliminating the advantage of having horses in the first place.
Cavalry fights by "shock and awe" - horse and rider weigh a lot, and horsemen may have a long lance (preferably longer than pikes of enemy infantry). After picking up speed, charging cavalry drive lances into enemy, disrupting enemy formation, and nearly immediately crashing into incoherent enemy unit. If enemy routs, they simply chase them down, if enemy stands despite loses, cav retreats to charge again. And again. And again. That's how cavalry fights - never stop, always in motion.
As such, Dwarves have no cavalry for simple reason that cav is useless in cities, sieges, mines, mountains, swamps, rivers, forests or any location which isn't kilometres of flat plains.
Since Dwarves usually are portrayed as not just fond of mining, but actually living in underground cities, cavalry is not useful for them. Their environment favours heavy infantry (light infantry isn't useful for Dwarves for pretty much same reasons as cavalry). Anyone who tries to take over their mines or cities, needs to fight on their terms: in cramped tight mine shafts, dark and cramped cities or built over caverns. Because of that, only use Dwarves would have for cavalry is for offence. If Dwarves tried to invade plains for any reasons, they would need cavalry, but usual portrayal shows Dwarves as either isolationist, or living in mountainous area.
Overall, depending on portrayal, Dwarves have literally no use for cavalry and little use for archers, crossbowmen and light infantry (unless they live in mountains and partially under, partially on surface, then they would have use for light infantry and archers or crossbowmen). Optimal heavy infantry weapon depends on details: Dwarves living on and below surface, would certainly have a use for pikes (pikes work well in tight formation), but living entirely underground makes pikes too cumbersome, turning attention to short weapons with anti-armour properties: staple warhammers and picks (or Horsesman's picks, just without horse) or perhaps maces. Short swords, like Gladius used by Roman heavy infantry, would be be very useful if apart from each other, Dwarves would fight underworld beasts which don't wear armour - short swords are especially effective with shield and in tight formation where short length is an advantage (long sword would be cumbersome in tight melee) but compared to warhammers, swords trade anti-armour properties for versatility.
As such, Dwarves have no cavalry for simple practicality: it's useless for them.
All in all, lack of cavalry, light infantry and archers is perfectly logical. Staple weapons are logical too. What isn't logical, is surprising lack of shields. Common image of Dwarf doesn't feature shield which makes no sense to me - shield and one handed weapon is an excellent combination for heavy infantry fighting in tight formation.
And here, for some elaboration of concepts, with images.
Why I say that pikes could be useful?
You want Iron, punk? We have it right here.
This image shows Macedonian Phalanx. Imagine something like that blocking the main entrance to the cavern. Or blocking the only flat passage through mountains. Rocky walls to your left and right, steel wall in front. Phalanx doesn't manoeuvre because those pikes ARE long but is excellent at spearing anything that tries to take it head on. If Dwarves live partially on ground, they would have use for this formation, even if they live in mountainous area, because you could block off bridges, tunnels or passages - but this would be the only function of this formation. I imagine speardwarves would be regarded with special reverence as first line of defence - protectors of mines and cities who never back off, blocking entrance with piling up enemy bodies.
Keep in mind that it would NOT be main force. It's extremely specialised, but very good at doing what it's specialised to do - blocking the path. Other types of heavy infantry would handle fighting on slopes, in cities after enemy breaks through etc. but a unit or two of those would probably serve as first line at the entrance. Perhaps speardwarves would actually carry swords or hammers and simply drop the spears to convert into ordinary heavy infantry if situation demands this.
Replace with halberds for much better manoeuvrability, late medieval look and ability to handle rough terrain at expense of reach.
That's Pope's Swiss guard. This guys might look silly now, but that's because they retain their ceremonial look. Church after all loves ceremonies. Back when Pope started hiring them, they meant business, Pope started hiring them specifically because they meant very serious business.
Shields at 70% and holding
Shield wall is a very old concept.
But it just keeps showing up.
Again...
...and again.
Ok, I cheated a bit, 2 last images are the same thing: riot police.
Shields walls are great in melee. Close range, packed space, short weapons which are easier to operate in such crowd. It even provides nice protection from arrows.
So, what are the drawbacks? Because it lacks spears, it's very vulnerable to cavalry (if you add spears you get Phalanx and lose the flexibility). It's also very vulnerable to firearms, so I guess shield wall wouldn't work so well if your world has significant numbers of battlemages. But if there's no cavalry and shields are enchanted against fireballs, this formation of heavy infantry will be very hard to crack in cities or passages and can work quite well in open terrain.
Historically, shield wall beat the phalanx through much better flexibility, which is why I said that spears would be useful but only in very specific tasks. I imagine that main Dwarven fighting force would be heavy infantry utilising shield wall, and either anti-armour pick or generalist sword.
Light infantry exists to chase off archers and screen heavy infantry. Kind of like infantry version of cavalry. Not very useful in direct engagement, but loose formation makes them hard to hit with arrows. Archers would often stand in front of heavy infantry, and retreat behind, just before enemy infantry gets close. Task of skirmishers is to run in front of heavy infantry to threaten archers while heavy infantry marches on. Skirmishers would often have javelins to throw at enemy before retreating and letting heavy infantry do the fighting. Later on, light infantry could try to run behind enemy formation for a flanking attack.
This formation wouldn't be useful in caves, mines and cities, but would be useful in surface part of Dwarven homeland. Unfortunately, shortness of Dwarves is a problem hard to overcome. I imagine that perhaps they would employ human mercenaries, or let some human villages exist on their land, specifically to have a source of skirmishers for surface combat (compare to Janissaries - Christian slaves, taken away by Ottoman empire at early age and trained as elite infantry).
In this scenario, Dwarves would pretty much use human skirmishers instead of cavalry - good enough at screening main force, and can always be re-equipped for underground fighting with standard dwarven tactics.