I love wacky questions!
OK, we need a removable object or our hapless test subject will have trouble eating and it must be made of surgical stainless steel or it'll start rusting very quickly. Stainless steel was invented in the early 1900s, so that's not an inhibitor.
Bolting things onto the human body is a bit more complex. Your first rod to help a broken bone was about the WWI time period but it didn't advance quickly. Bolts were first used around the WWII period, so it's plausible — unless your happy with the entire steampunk motif, in which case it's mandatory.
So, let's say the bottom end of our technology is the 1950s or early 1960s. We're going to bolt a stainless steel infrastructure to the upper and lower jaw areas between tooth roots to hold what would most likely be a single-shot also stainless steel firing mechanism. Various hooks and latches will be used for removal, leaving the bridle-like infrastructure in place to swallow around. It's inefficient, difficult to aim, definitely 1970s James Bond in its pure silliness, but it's plausible.
I wouldn't activate it with my tongue, too much liklihood of blowing a hole through my own teeth. Activate it by opening the mouth wide. You'll feel the tension as you're opening your mouth to the correct width, and finally Click! Bang! And then...
The real problem
The real problem is recoil. The human arm is wonderful at absorbing linear energy. It's almost as if it were designed to punch people in the nose. Your neck on the other hand, was not designed to withstand a lot of lateral energy. Let's call it whiplash.
Now, a .22 calibre bullet doesn't produce a ton of energy compared to its larger siblings, but it's there.
The acceleration-deceleration forces which cause whiplash injury are sufficient to permanently disable you. Even in a low speed rear impact collision of 8 mph, your head moves roughly 18 inches, at a force as great as 7 G’s in less than a quarter of a second. The Discovery space shuttle is only built to withstand a maximum of 3 G’s. (Source)
7G is 68.6 m/s2. In a quarter of a second that's about 17.15 m/s. A .22 bullet has a muzzle velocity of around 914 m/s. The average adult head is 4.5Kg, the bullet is 2g, that's a ratio of 4.4e-4 so the head is going to move, what, about 41 cm/s1 for more-or-less that quarter second. That's a long way away from the 17.17m/s we need for a dibilitating injury. It'll be a pain, but our intrepid agent of evil will survive.
Which means your real, real problem is...
Accuracy. I can't think of anything less accurate than a mouth gun. This is point-blank wet work only unless you've been distracted by, well, her... Let's just hope you're sent to assassinate a lady.
1 This is my favorite way to quickly estimate the transfer of energy between two objects. It's inaccurate, but good enough for government work.