This system would leave large numbers of men unmarried, which has historically led to an increase in violence.
Why would unmarried men become violent?
In most societies in our world men are supposed to only have sexual interactions with their married wife (or wives). "Cleaning the pipes" on their own is still seen as inferior solution and was portrayed as a health risk in the past to enforce sexual abstinence outside of marriage. There's an obvious statistical correlation: Men without any sexual outlet tend to become more violent. In countries with prostitution illegal and a societal stigma (or even laws) against premarital sex the rape rates are significantly higher than in countries without this. (Thanks to ArtificialSoul)
If you remove this barrier to sexual satisfaction for unmarried men you could remove a lot of frustration and potential for agression. Being invited into the bed of a noblewoman is a high privilege, but even the lowest ranking man should have a place he can go to satisfy himself.
Depending on how you want to portray your society, you could have the typical female sex worker, but it seems that male sex workers would be more realistic in your setting.
- Could be regarded with the highest respect due to the stabilizing services they provide for their society
- Could be the second born, third born... who are selected to perform services (like third born sons where sent to monestaries in the past)
- Or they could be expelled and outcast daughters
- Could be forced to work until they're married
- Could enjoy higher previleges and more rights for their services
- Could use the service as a means to promote themselves as satisfying husbands
If you find the idea of sex work repelling, you could install a culture of free dating in your society. Have dedicated places (like gardens) on each clans property where everyone willing to share a bed comes to at night. Women have free choice, of course, and those men who were not selected for the night find a male partner. These places should also exist in public areas to give unmarried men the same opportunity.