You really want to have a weird set of weapons here.
Lets look at them in more detail, especially regarding their use case and time periods.
Firelance
The classic single-use paper-barrel firelance was in use mostly from 950 to 1300 in Asia and sporadically used in Europe in the 1400s as a mounted weapon. These things where terribly ineffective at actually hurting people but they made a mighty bang. Their main purpose was to scare enemy soldiers and put them in disarray. These were never really used as "the standard infantry weapon", but you'd rather have a few dozen of them distributed among your troops to mess with enemy formations.
Hand cannon
These weapons are now first proto-guns that are actually used for killing people. They appear around 1128 and by 1375 they have flash pans and soon they also have shoulder stocks, making them quite different from the primitive hand cannon that you have in mind. The main reason to use hand cannons over other weapons is their ease of use. They are terrible compared to weapons like a longbow or even a crossbow, but you can hand one of these to someone who has never used them and they will have moderate success with it, compared to a longbow, which an average person can not even launch an arrow off.
Portable mortars
The image you linked is actually a hand cannon being fired from a stand.
Heavy mortars
The first usage of mortars in combat happened at the 1452 siege of Constantinople, which was rather unsuccessful since the mortars where primitive and extremely heavy, making them terribly immobile. The first actually useful and more mobile mortars were used 1719. So we are talking about a few hundred years of further advancement over your other weapons. So while there was an era where hand cannons and firelances were conceivably used at the same time, that's not so with mortars.
As for other kinds of cannons, they appeared earlier (starting from 1380) but, due to their heavy weight and horrible mobility they were mostly used stationary, so when defending in a siege. Until the 16th century, siege weapons like the trebuchet were much more useful when attacking fortifications than cannons were. They only started being used in the 18th century. As with mortars, the golden age of canons was much later than the golden age of firelances and hand cannons.
I could not find any proper source for the image you posted, but it seems to be something from the 16th century or newer, by my guess.
Summary
So, from a technological standpoint we are talking about weapons that differ vastly in their advancement. Firelances were the very start of gunpowder weapons while mortars and cannons appeared half a millennium later. Kind of like throwing a mechanical targeting computer from the 30s in the same category with a modern smartphone.
What mostly limited the advancement of gun development was that many metalworking and general manufacturing processes had to be invented first before they could be used in guns. For example, steel production in Europe only came up in the 17th century. Before that you had to use iron. To have the same strength as a steel barrel, an iron barrel needs to be much thicker and thus heavier. Mechanisms like the flint lock need to be manufactured with a very high precision to work reliably. So if you have these kinds of manufacturing processes available because your society is very advanced, then you will obviously have them available for gun-making as well.
So if you have a 16th century mortar, you could also expect to have a 1411 arquebus or even a 1475 matchlock arquebus. Both look much more like a contemporary rifle than like a hand cannon.
So how to apply this?
The combination of weapons you describe is not a very good fit, since they need very different manufacturing processes available. From a technological standpoint there is no reason why these weapons would be used together.
The way you described the usage of especially the firelance does not fit at all to what they were actually used for.
I would probably rethink the weapons. Drop the cannons and mortars and replace them with grenades that are thrown by trebuchets or similar devices. That would fit much better with the firelances and hand cannons. Maybe add firearrows as well. That would make for a much better fit. Also, drop the firelance as the primary weapon and have them use it like it was used in reality. Give them spears or lances as their primary weapons.
Now, with the weapons sorted in the right time frame and usage, why would they use those ancient and useless weapons if they have access to manufacturing processes that allow for much better weapons? Also, why don't they use better non-gunpowder weapons like a longbow?
Since there is no good military or technological explanation for that, there needs to be another reason for that. I have a few approaches to that, but most of them have their caveats.
They don't need anything better.
They are so technologically advanced over all their enemies that even after a millennium firelances still outclass everything their enemies have to offer. The caveat here is that this only works if your ork nation is completely isolationist so that their weapon designs (and even the concept of how such a weapon works) do not leak to an enemy who could improve on these designs. Also, all Orks need to be pretty lazy and not a single youth can be interested in things that go boom enough to create better designs (remember, even a decently built potato cannon is much better than any firelance ever built).
There are no enemies.
Orks are perfect when it comes to diplomacy and are really important trade partners. This only works if they are mainly good at processing goods and don't have any natural resources worth invading for. Basically, they are the Swiss.
They have massive armies.
Orks breed like rabbits and can afford to loose warriors 10:1 when in combat. There are just so many of them that it does not matter if they loose tons of warriors to enemies with vastly better weapons. Also, dying to the hands of enemies has to be a cool thing in the Ork society. Or they have overpopulation problems and that's their form of population control.
Religious or social constraints limit them from using other weapons.
Their God once said "thou shalt only kill thine enemies with crappy weapons" and they stuck with it. Requires that at least a huge majority of their soldiers would rather die in horrible massacres than offend their God and/or his priests. Everyone in this nation has to be a real fanatic.
Certifications.
Ork-land is the worst bureocratic distopia ever. Everything that might be able to take a life needs to be certified and the process for that can take centuries. Only weapons that make it through this process or are older than the process are allowed to be used. Firelances and hand cannons are older than the certification process and are thus allowed. Mortars have only recently been approved, even though other countries don't even use them anymore. This is the only thing I can come up with that might even work with your weapon combination and -usage.
It's a Marvell movie.
Do it like they did it in Black Panther and just hand-wave it. There is no reason other than that you as the author like the concept. Depending on the kind of story you are making that might be enough.