If it is possible to manufacture a tiny, functioning gun about the size of the smallest baby shoe, what would be the limit of its firing power?
Since this is the worldbuilding site, I presume you are asking for a character in a work of fiction. A lone gunman or a secret agent with the full support of a Q department? Bad guys might find help in the tribal areas of Pakistan.
The .25 ACP round is a bit over 2 cm long and it can be effective from a 5 cm barrel. An automatic .25 pistol can be built just over 10 cm long. A gun like that may be underpowered as a weapon, but there is no doubt that it is a weapon that can fire deadly shots. There are disguised weapons of a similar size.
Both of my examples are are larger than your stated size requirement. On the other hand, the .25 ACP is more than a century old. Modern manufacturing technology should be able to bring the cartridge down to less than 2 cm, and it should be somewhat effective from a barrel of 4 cm or thereabouts. A single-shot pistol build for minimum length should be possible at 6 cm or less. The grip would be a problem, and the effective range would be limited to a few metres.
The smallest gun that I have ever seen was a .22 cal assassination weapon built during the occupation of France by the resistance. It was disguised to look like a ladies compact (see picture; this is an example of what it would have looked like); opening the mirror cocked the action (and gave the shooter an unobtrusive way of lining up the target), the case was then closed and (if I recall correctly) the weapon was fired by pushing down on the top of the now closed case.
As a weapon firing a .22cal round from a barrel only a few centimeters long, there would not be either a lot of energy or accuracy, so this would be something used at point blank range. While I don't remember if it ever was actually used, I suspect that the odds of being able to both kill your target and escape would be pretty low. The compact/gun could conceivably been made smaller, but there needs to be a minimum size for the shooter to actually be able to manipulate and aim the weapon.
The only way to get much smaller would be creating a weapon with high tech ammunition. Imagine mechanical mosquitoes carrying ricin toxin, sea snake venom or some other noxious substance. A few could be fitted into a thin glass or plastic tube. The shooter adds a few drops of pheromone or other identifier (or downloads the target specs from an iPhone) and opens the end of the tube, allowing the mechanical insects to fly out and attack the target. While not technically a firearm, it is probably about as small as any practical weapon could be.
Though your size analogy is disconcerting, very small guns do exist.
This gun apparently has the following specs:
The SwissMiniGun's 2.34mm-caliber ammunition travels just under 400 feet per second. Its bullets pack a punch of about 0.71 foot pound of energy [source: SwissMiniGun]. By contrast, the Remington 300 Ultra Mag round carries 4,220 foot pounds of energy with it [source: Remington].
It could shoot needle-like projectiles, in which case many could be stored within the gun. Also, if they were poison-coated, even more could be stored - the projectile could be shorter, as the damage would come from a toxin, as opposed to physical damage, which is affected by the size of your projectile.
Or maybe not - it'd still need enough inertia to reach the target.
On the most basic level a gun is a pipe that fires bullets, so if the shooters comfort isn't a huge concern you can fire most any round that is small enough to fit your spec.
The shooter's comfort becomes an issue at this scale though, large calibers in tiny guns tend to transfer a bit more recoil to the hand and keeping a grip can become an issue.
I'm by no means a gun guy, but it seems to me it shouldn't be any less powerful than another gun that takes the same caliber bullets. The force of a gun comes from the energy of the gunpowder packed into the bullet, distributed along the tiny cross-section of the bullet.
Gun guys could tell you, but I presume the short muzzle length reduce accuracy. It might also reduce force if explosion reaches the end of the muzzle before the bullet has absorbed the majority of its energy.
But honestly, the biggest design concern would be handling the recoil. There's simply not much room for a grip. Maybe you'd sacrifice accuracy by removing the grip and just using a flat metal plate, so the recoil would be distributed across the entire palm of someone's hand?
As many have said, such a small weapon means a low bullet velocity--but only if we confine ourselves to traditional bullets. Instead, how about the Gyrojet "gun"? It uses what look basically like ordinary bullets but they're actually rocket boosted.
Unfortunately, it has an achilles heel in the form of accuracy--as it leaves the barrel slowly it's much more vulnerable to wind in the early part of it's flight. It also sort of has a minimum range--it's still boosting for part of it's flight and if it hits within that distance it won't have it's full power yet.
If you want a single-shot weapon with maximum bang you could scale up the gyrojet round and pack a small explosive charge in it. Use a slight delay fusing so it penetrates before detonating and you have something exceedingly lethal with any central body hit.
While I do not know I would expect the rocket to leave a big arrow pointing back at the shooter. Your assassin would not escape.
While this is technically not a gun, a grenade would be a good way to estimate the absolute maximum damage something of that size could have with conventional weapons tech.
The V40 is a grenade that comes in at “ 6.5 centimetres (2.6 in) high, and 4 centimetres (1.6 in) in diameter - approximately the size of a golf-ball”, and weighs in at 134g, about the weight of an orange.
Upon detonation, 326 squares are fragmented and shot out, giving a lethal explosive range of 5 meters and is considered dangerous for up to 300 meters. Compare this to the M67, a grenade weighing in at 400g, at 2.5 by 3.5 inches, which has a similar explosive radius.
Of course, the destruction is mainly given by the speed at which the fragments shoot out, but note that the grenade also consists of the safety mechanisms and trigger. The explosion and fragments are considerably more dangerous than any bullets from a gun of the same size, primarily because the delivery mechanism is the human themselves. Something small like the V40 could also be easily disguised.
If it has to be done, I’m sure there is some method of attaching the grenade to a one-time-use rocket, allowing it to be fired off like a firework, at the expense of accuracy and possibly a few fingers.
If it can be aimed properly, and triggered to explode somewhere close to the target, the squares would utterly shred the target, assuming it is some form of mammalian species.
Thus, a grenade would probably be a good point of reference on the maximum payload a gun the size of 2.5 by 2.5 inches can deliver.