Grade matters.
The strongest grades of aluminum are, after thorough heat treatment, significantly stronger than commodity steels. They are used to build aircraft, and, handling that piece of metal, you wouldn't think of it as the aluminum most of us are used to. It's stiff, springy, and doesn't bend at all.
Realistically, ancient or medieval technology will not allow for the kind of heat treatment high-strength aluminum requires. It was understood for steel, but it's just a different technology level to do anything other than simple quench hardening.
Soft aluminum is still useful. Most laptops are made out of 6061-O aluminum, not heat treated, "good enough for customers". It would be useful as a general purpose medium-strength metal, similar to copper and bronze, but at lower density.
Replacing lead would be very desirable, but, unfortunately, not very realistic. Lead has a yield strength of just 5 MPa, compared to aluminum's and copper's 80-100 and steel's 200+. It's not possible to cold work aluminum the way you can with lead.
None of the military uses would be revolutionary in nature. However, aluminum could be useful for a variety of tools and devices.
The most immediate military application would be shields. Steel reinforcement tends to be heavy, aluminum over wood is just perfect, and easy to use. Armor uses in general, to the extent that manufacturing is possible. And generally, structural uses where high strength-to-weight is required.
Personal experience: when I was learning the ropes of swordsmanship, I had a partial set of aluminum armor. Completely historically inaccurate, not even trying to look authentic, but it was way easier to carry to the training grounds and around than steel. And protected me from blows just as well as much heavier historically accurate steel reproductions did.
Even conventional low-strength alloys perform well for armor, where thickness is king. Bending stiffness scales as cube of thickness. At 80 GPa, 3mm of Al offer similar stiffness to 2mm of steel (200 GPa). At the same time, 3mm aluminum is easier to cold-work than 2mm steel, and only half the weight.
However, sharpened hard steel, as used in combat rather than training, would be effective at piercing soft aluminum, reducing the utility of aluminum armor. Still, the latter would be very useful for areas where bending rather than piercing is the expected failure mode. That's mostly the limbs. Basically, the smaller the armor element, the more use for aluminum.