Fireworks are designed for beauty, not for damage.
Many years ago, before lighting became so practical, mankind was mostly in the dark at night. Kings and other land holders would hire a pyrotechnician for celebrations and as a demonstration of their power. The display of fireworks would dazzle and delight people and remind them subtly the ruler had the power to illuminate the night sky. This is why so many celebrations in our society are celebrated with fireworks. Independence Day, New Years Eve, sporting events all have fireworks that light the sky. Beyond offering a demonstration of power, fireworks make poor weapons because they emphasize color over power and that color actually retards the destructive potential.
Fireworks and destructive weapons like a cannonball are both powered by black powder, but they use it in remarkably different ways. With fireworks, they use black powder mixed with powdered metals for color and other pyrotechnic compositions for an emphasis of color over effect.
Rockets Red Glare
Lets take the example of a rocket. A well-designed pyrotechnic rocket will used compressed black powder and titanium to create a trail of sparks as it powers the rocket into the air. This makes it a terrible offensive weapon because as it streaks into the air, it creates a trail that leads to the launch location and exposes your position.
Cannonballs Versus Mortar Balls
Fireworks shells are known as mortars. They are designed to break, not to explode. A 3" shell in a tube fired from the ground is designed to break approximately 3 seconds and about 250' from the launch location. That doesn't give you much distance from your enemy. This is opposed to a cannonball which is designed to be propelled up to tens of miles from the launch point.
Fireworks are not designed for launch accuracy. If you see a mortar shell launched, they tend to wobble as they head into the sky. They are quite light in weight, with a a body composed of a thin layer of cardboard and paper are propelled with a few ounces of black powder. They move about 33 feet per second, which is not enough inertia to cause any serious damage.
To get them to cause damage, you're better off starting from scratch and creating devices to do damage. As an example, the cannonball. A 10 lb cannonball made of iron in a rifled cannon barrel can be fired with 10 lbs of black powder will have the inertia to penetrate thick walls. If you used a similar amount of black powder to launch a pyrotechnic mortar, it would most likely rupture in the cannon barrel. One of the issues designing really big shells (16" and larger) is to make the walls strong enough to handle the force of the gas expanding to send the shell into the sky. I have seen them flowerpot just above the lip of the mortar which means days and days of work put into making the shell ended in failure.
It's in the stuffing
The stuffing in fireworks are round balls or cubes called stars. They consist of black powder, a color or effect, a binder and slowly rolled to build a ball or pressed to look like a cube. It all depends on the effect you desire. Because of the binder and compression, they burn slower than the black powder would burn if you are looking for maximum expansion of gas, which is what causes the shell to break as a pyrotechnic device. It can take days or weeks or months to make enough stars for a full show. In that same amount of time, you could create effective devicess which are designed to create damage.
As a licensed pyrotechnician, I feel mentioning the facts about pyrotechnics will improve your story. To sum it up, Roman candles will do no damage. A mortar could kill you, but chances are it would just injure someone and on impact, bounce and go a different direction and if you're looking for guns powered by fireworks, just use straight black powder, like we did for a thousand years.