Consider a society that is both matrilineal and non-monogamous, such as Patrick Rothfuss' Adem from The Kingkiller Chronicles.
For those unfamiliar with the work, there are basically two factors at play:
No Cultural Aversion to Sex
When an Adem is feeling frisky, they seek to fulfill that need as a matter of course, and have few aversions as to whom they use to that end. One of the Adem's nearby friends might even offer to "help"!
This invariably leads to lots of non-monogamous sex. (Perhaps a tendency towards monogamy later in life, but irrelevant for the example and ultimate question)
The Myth of the Man-Mother
Women naturally ripen like fruit, and give birth to children. Men have "no" effect on whether or not a Woman ripens.
This means that any given Adem has no firm knowledge of whom their father is (though they don't care and don't believe in "man-mothers" anyway).
It is quite possible that Adem could mate with their own half-siblings, cousins, aunts, uncles, etc.
Is this likely to cause their inhabitants any harm due to accidental inbreeding? Or is the sort of thing where as long as it doesn't happen too much (and since the culture isn't intentionally inbreeding), it's a moot point?