Yes... and no...
A volcano explodes (very simplistically, this is the problem) with a force. Apply the equation F=mA. We know the force of the volcanic explosion, we know the acceleration of Earth's gravity. Voilà! We know the mass of the rock we need to plug the eruption.
Sounds simple, doesn't it...
Outrageously ignoring the fact that volcanic eruptions occur for a number of reasons, we'll simplify the discussion to assume Mt. St. Helens is indicative of the complexities of the problem. And we can sum up a lot of that complication with one word...
The eruption of Mt. St. Helens was preceded by a number of earthquakes of various sizes, ranging from itty-bitty to magnitude 5.1. Earthquakes have the nasty habit of softening up and shifting around rocks, faults, vents, and everything else associated with volcanism.
But the fact of earthquakes isn't all, where they take place is also a problem. Some are deep in the earth, others are shallow. Some are miles away, some are right under the honking volcano. And they all add up.
And we're not done
Just to make things ugly (remember, I'm already simplifying by ignoring types of volcanism), you have the geological makeup of the surrounding area. Do you have a really tall mountain with thin walls? Or a really low one with thick walls? Do you have a big throat or a small one? Is the Earth desperate to pop this proverbial zit? Or is it just trying to clear a little mucus?
Not to put too fine a point on it, but you're trying to fix the Earth's "grumbly tummy" by putting a billiard ball in it's mouth. The billiard ball is certainly more mass than whatever might come out of the stomach,1 but if you tried it yourself, you'd likely be surprised to discover that it didn't stop much — in fact, it usually makes it a lot worse.
So, yes, for some (few) eruptions dropping a big ol' rock on top will stop it. For the rest... you'd better have evacuated the area, first.
But it'd sure look cool, wouldn't it? Kinda like when they tried to get rid of the beached whale by blowing it up.
Oh, and just a reminder, musket rifles use the same basic premise of putting a billiard ball atop a grumbly tummy. And what goes up....
1 In my metaphor. It's my metaphor, after all.