If the gem is only encrusted upon the steel as ornament, it will just be dead weight. But such gems are pretty light, compared to steel. Some quick googling has the density for emeralds at around 2.76 grams/cm3. Steel, on the other hand, is around 7.8 grams/cm3. For comparison, the density of water is 1 gram/cm3.
Now, the emeralds may protect you metaphisically - I won't argue about that, you world, your rules - but if you intend for them to protect someone mechanically, your knights are in trouble.
A lot of people are going to tell you that emeralds and diamonds are awesome because they are hard. But that DOES NOT make for armor material.
Armor protects you in two ways. It either deflects blows, or absorbs impact for you.
Now, I don't see your emerald deflecting a mace, for example. It won't have the shape for it. So let's try absortion.
Metal absorbs energy by changing shape. That's why cars are are built with crumple zones, so that they absorb the energy of an impact. In the 20's cars didn't have these, so in a car crash whatever was inside them (like people, for example) would take all the energy of the impact (and die). Metal armor works just like that.
Steel is good for armor because it has a good mix of hardness and toughness. Yes, toughness, that's a technical term. It means:
(...)the ability of a material to absorb energy and plastically deform without fracturing.
Now, diamonds, emeralds rubys etc. have the wrong mix for it. They are too hard and are not tough enough. Let me explain:
Too hard: they will not bend nor change shape in any other way to protect you. If you could build an emerald helmet and wear it while I hit your head with a hammer, that emerald will happily transfer all the energy of the impact to your cranium. You'd fare better if you were using a metal elm instead.
Not tough enough: if people keep saying that emerald would make good armor because it is very hard, I will go hulk-mad and hit you on the head with even more force. If you are still wearing an emerald helmet, at some point it is going to shatter like glass. Not only you took all the kinect energy for yourself (because that precious stone will never take a hit for you), you now have shrapnel in your cranium.
So please, don't trust your life solely on those stones, ok?