Let's build on my comment
As I said in my comment, one of my favorite books is Philip Frances Nowlan's Armageddon 2419 A.D. It's the book that started the entire Buck Rogers franchise — and one of the interesting things about the story is the use of rocket guns.
That novel was written in 1928. The idea has been used in SciFi for (at least) nearly a century — and yet your question is being dismissed as implausible to impossible.
Buck Roger's rocket gun was specifically useful because (remember, 1928) they had jet packs. Oh yeah! Jet packs! The epitome of science fiction! And when you're hovering in the air the recoil from a rifle is bad juju.
Recoil has been a problem with firearms since day one. It limits how much ordinance can be brought to bear. It affects how quickly the next round can be fired. It affects accuracy. The history of firearms is also the history of the quest for recoiless arms. And not for space, but for use right here on terra firma.
Consequently, rockets have been and continue to be used in real life to deliver ordinance that would normally cause so much recoil that the operator would be seriously harmed.
Which means your idea has a basis in plausibility: recoil.
Next: are bullets cheap? You betcha! But let's examine that idea. Right now on the shelf at WalMart (well, maybe, ammo shortage and all...) a rifle round is \$1–\$2 per round. I have no idea what the military pays for ammunition — but they use a whomping lot of it — so let's assume about 30% of civilian cost. Let's call it \$0.50/round. And @JamesQF in a comment states that in real combat, 50,000–300,000 rounds are expended per soldier killed. Average: 175,000 rounds (sounds a bit high James, but I'm willing to believe. Do you have a source?). That's $87,500 USD per soldier killed! (James? You sure about those stats?)
What that analysis tells me is that if a soldier can be killed in one shot by a personal smart missile, and that missile cost $42,000 USD, the government would jump at the chance to implement the technology.
Which means your idea has a basis in plausibility: economics and efficiency
Now, we are in combat, whether it's here on Earth or up in space. @KerrAvon2055's answer tells us that the average combat range for the police is three yards. I'd like a source for that statistic, too, because police in the U.S. are trained to the "21-foot rule," (which is being disputed after all these years, but work with me), which basically says that the bad guy must be 21 or more feet away or they can rush you before you can unholster your gun and shoot them. 21 feet is better known as 7 yards, which is why I'm having trouble with the FBI reporting that the average engagement distance is only 3. I'm happy to accept being wrong about this, but it would mean a lot of bad guys are being dragged out of cars or something to bring that well-trained number down.
Now, to be fair, 7 yards isn't exactly a lot, either. But it brings up a point. Could the police use this kind of technology? Absolutely! Suddenly they have the ability to fire a shot at a running suspect and have a hope on Earth of actually hitting them. Better still, the payload could be an anesthetizing gas rather than a high explosive or a simple tranquilizing dart. Personal smart missiles could believably revolutionize police (and military, and space) combat.
Which means your idea has a basis in plausibility: combat conditions
And one more thing, the idea of a target-and-shoot (or, better yet, a point-and-shoot) personal smart missile has one really big appeal: minimal to no collateral damage. Buildings and innocent bystanders don't get shot up in the firefight. Personal missiles would be desirable simply because it allows a nervous police/military to worry less about the mess they might cause if they pull a trigger (not that the military usually cares about the mess it causes... but the politicians greedily looking at that city or space station and linking how valuable it would be if it weren't all shot up might).
But is the technology believable? I mean, pencil-sized missiles?
Folks, this is WorldBuilding.SE. If we can't imagine the ability to miniaturize a technology (something that's already been done for this technology anyway...), then who can? I'm a big believer in the idea that unless the OP says otherwise, the fact that we can't do it today has no bearing on the believability of the idea in the future. But, what would you need to make it work?
You could rely on thermal tracking. This would work in space. It would technically work on the ground, but @GrumpyYoungMan brings up a point: you need to move around things like trees and bushes on the ground. Could that be done? Sure! We do that already with surface-to-surface missiles. The problem really isn't the trees, it's getting the missile over the top of them in a meaningful way.
In an urban setting there is the problem of buildings and narrow streets. However, with every passing day, Google Maps becomes more accurate. It's easy for me to believe that the efforts of cartographers and database engineers will result in accurate enough 3D maps of, frankly, all surface locations such that the missile would need little more than to know its starting point and a means of tracking its target and it would know where to go. (Watch that little bounder use a subway tunnel to cut off a fleeing assailant.... It's scare the crap out of the public, but it'd make for a cool story.)
But could a small missile maneuver in the big city? Once again, why not? A small missile would have an easier time (assuming all the aerodynamic math checks out) than a large missile. Add satellite tracking (or cellular tracking, if you want to extend a terrestrially-based technology) and we have a winner.
Now, to be honest, there would be limitations — but none that I can think of that would cause bullets to be the preference.
You have limited range, so the purpose of the personal (e.g., something shot out of a pistol) missile is interpersonal combat. Taking on an airplane would still require something bigger to hold all the fuel and the propulsion system necessary for the speeds and distance involved.
You also have limited payload. The physics of "boom" don't appear to compress exceedingly well. On the other hand, Asimov's Foundation series imagined nuclear reactors on a necklace, so I suppose one could imagine a nuclear bomb on a personal missile. And you might need one if you're up against powered armor on a space station.
My conclusion? I think it's reasonable to assume that a story or world that supplanted bullets and other slug throwers with missiles of all shapes and sizes is completely believable. In fact, since it's already been done, I'm having trouble understanding why anyone would suggest that it's not a fine technology for your world.