Inbreeding does not automatically result in birth defects or cripples. It's the method used to produce "purebreeds" and it's no different between horses, dogs and humans; it will increase greatly the chances that any negative trait that is present in the genetic stock, and is normally masked by genetic variability, will surface and breed true - resulting in birth defects, stillborns, and cripples (this plus controlled reproduction of the healthy individuals only, will allow to weed out genetic defects and select for desirable traits until they "breed true").
For example, the father is XY and the mother is xX. This is a typical situation for haemophilia. Neither parent is affected, but the mother is a carrier. The children might be Xx (carrier again), XY (unaffected), xY (affected), XX (unaffected). Now it is possible to get a xx offspring in the next generation from the Xx and xY siblings. Breeding the apparently healthy individuals with one another (Xx and XX vs XY) will yield a healthier second generation and allow surmising that one of the two females is unaffected, and stop breeding the other as well as the other's offspring.
Of course, the more the latent defects, the more difficult and lengthy the process (it's little use to have a child which is healthy for traits A and B if they die in their infancy because of trait C).
But if those defects aren't present, they can't come to the fore. In your case, with several possibly unrelated wives, it is unlikely they would come to the fore for several generations; the current stock has probably been vetted enough in the past, "sword in the heart" style.
There is such a situation, if memory serves (minus the sword part), in Charles Stross's The Family Trade, where marriages are organized to maximise genetic diversity within an inbred extended family; the whole marriage business is called the "wreath" and is supervised by the aunts and grandmothers.
(For humans, you might have to take the Westermarck inhibition into account)