There's several examples of this trope being averted (Harry Dresden, Mustrum Ridcully, Belgarion, Sparhawk, Gandalf etc) but if we look at the more specific context in your question:
Edit: let's just restrict this to medieval technology as it is the most common used in fantasy anyway, but instead of gun there still were repeating cross bows and daggers which only take a fraction of a second to throw so the harry potter example still applies, gun or dagger the difference is little they are both light, small, easy to carry and quickly kill someone.
Repeating crossbows, especially not ones that can be used single-handed (pistol crossbows) aren't that common in medieval fantasy settings and without at least a semi-automatic mechanism they can't be re-cocked single-handed anyway so you're not much better off than a one-shot crossbow anyway. Accuracy using it single-handed isn't going to be great either.
As for knives/daggers - throwing knives quickly and accurately enough to kill someone is hard, and that brings me to perhaps the main in-universe reason why they don't use side weapons. To be able to kill or incapacitate any opponent with a mundane weapon reliably enough to bother carrying one around you need to be reasonably proficient with it.
Take your Harry Potter example.. just how proficient do you think an 11 year old school child with zero weapons training going to be with a pistol crossbow, dagger or even a gun for that matter?
Getting good enough with a weapon to use it takes practice, hours and hours of practice. And since in many medieval-esque fantasty settings so does magic then mages simply don't have the time to spare to train with mundane weapons.
Out of universe of course there's always the question balance to consider. You have a mage who can not only blast fire at you but is also a crack shot with a pistol crossbow or an expert knife-thrower then you risk them being a bit over-powered and boring. It can be done but you have to think of another way of making them vulnerable.
EDIT: Updated to respond to the comment below
How proficient? Kids carry guns and knives around all the time and plenty of people get killed. Even the most unskilled person can point a gun and pull a trigger or stab with a knife. Training will make you better at it but it's still deadly with zero training.
At three metres, with ~1.5s to react, draw, aim, fire and have a decent chance of hitting? Sure the real top-flight people can pull that off. Novices? No chance.
Let's assume they have the gun/pistol crossbow out, ready and aimed in the enemies general direction before the 1.5s starts (although if they had that much time advantage they could have started their own spell surely) then you have enough time (~0.25s to react and even at 100fps the bolt will cross the distance in ~0.1s), will it hit? Maybe, it's certainly not reliable though.
Untrained people are really not very accurate at all and would you trust your life to the idea of making that one out of one shot? Especially in a life or death situation with the time pressure on?
Kids in gangs or whatever carrying pistols aren't exactly comparing apples with apples either. It's a fair bet that they will at least have had some practice with it and even then the chances of them actually hitting what they're aiming at is slim. They are usually killing each other more by luck then skill.
There's nothing inherently wrong with the idea of equipping mages with mundane weapons - the proficiency angle was a way of explaining why the OP's notion that Harry Potter could have just solved the whole thing in Book 1 by the simple expedient of packing heat doesn't work.