I also agree with John Dallman that your conditions are not enough to get the kind of selection you are looking for. However let me suggest some changes that might achieve what you want.
- Reproduction between castes is vanishingly unlikely. (It's not enough that marriage is unlikely if you race's behaviour is anything like humans.) This means that any genetic traits that arise in one class don't transmit to another. Effective and easy birth control that was usable by either gender might help with this.
- A few highly valued traits in each caste. So valued that anyone exhibiting these traits from a different caste would be instantly snapped up as a mate in the caste that desired them. Or maybe anyone with that trait is adopted into the caste.
- The castes are originally different races. So instead of the Normans invading and conquering Britain, it was the Moors, or North American Indians.
- Any child showing characteristics of the wrong caste is immediately killed as an abomination.