The energy released from the meteor impact is going to come from the meteor's kinetic energy. As the meteor comes to a sudden stop as it hits the city, most of that energy is going to be released into the environment (while some will be "wasted" in annihilating the meteor).
The kinetic energy of a body is $ E=\frac{mv^{2}}{2} $. So we have two variables, mass and velocity. From the equation we can see that doubling the mass of the object, doubles the energy, while doubling the velocity, quadruples the energy. (If we get into relativistic velocities then mass dilation becomes a factor too but let's ignore that for now).
So the question of how big the meteor needs to be is very dependent on the velocity of the meteor and how much energy we want out of it (with the proviso that the meteor is sufficiently large that it wouldn't completely burn up in the atmosphere before hitting the target city).
For illustration, lets assume we want a destructive force similar to the bomb dropped on Hiroshima. That had a yield of approximately 16 kilotons. In joules, that's about $6.7 \cdot 10^{13}\;\text{J}$. So that's the kinetic energy we're looking for from our example meteor.
To keep the math simple, lets assume that our meteor is moving at $50,000 \frac{\text{m}}{\text{s}}$ (which is within the velocity range of 'normal' meteors).
Substituting the known values, we get $6.7 \cdot 10^{13}\;\text{J} = \frac{1}{2}m \cdot (5 \cdot 10^{4}\;\frac{\text{m}}{\text{s}})^{2}$.
Solving for $m = 2\cdot \frac{6.7 \cdot 10^{13}\;\text{J}}{(5 \cdot 10^{4}\;\frac{\text{m}}{\text{s}})^{2}} = \frac{1.34 \cdot 10^{14}}{2.5 \cdot 10^{9}}\;\frac{\text{J}}{\frac{\text{m}^{2}}{\text{s}^{2}}} = 53,600\;\text{kg}$ or $53.6$ metric tonnes.
The actual volume of the meteor will depend on its density. For example, Iron has a density of $7850\;\frac{\text{kg}}{\text{m}^{3}}$, so a $53.6$ tonne iron meteor would have a volume of just $6.8\;\text{m}^{3}$. Thats equivalent to a sphere just $2.35\;\text{m}$ in diameter, which is tiny in cosmic terms.
If you want a larger amount of damage to cover your theoretical target city, you simply increase the size of the object or its velocity or both. Or you simply hit the city with a number of smaller meteors.