Amid a Sword and Sandal era, my protagonist comes from an ancient and powerful family who has lived in a castle for many generations. The problem is that castles shouldn't exist yet, at least not for humans. The protagonist's family is elven and humans and elves seldom interact at this point in history. BUT, this particular family oversees a village of humans that are something like serfs (but they are allowed to leave for certain reasons like trade).
Supposing that the castle is of elvish origin then, and supposing that magic exists, what would prevent humans from learning from and replicating this "new" architectural technology for themselves? Humans shouldn't start implementing this sort of architecture for another few/several hundred years and keep in mind that the castle has already existed for thousands of years without being replicated by humans.