Nerves work by changing electrical potentials. That could be the basis for your fence.
Living things have electromagnetic signatures; some fish can detect this and use it to hunt. Our brains and nerves work by electromagnetic potentials which is why electricity can knock us out or cause our muscles to move uncontrollably.
Your field is not a static thing like a fence or a wall. It is actively regulated by a program. When something with the electromagnetic signature of a living being crosses into the field, the field changes. Magnetic fields are produced which induce electrical fields within the organism. This can then do pretty much whatever your story demands. It might cause you to fall down with a seizure. Your muscles might be paralyzed. Your cerebellum might be so impaired that your coordination disappears and you wiggle and flop your way back out of the field. Or more subtle things - it could affect perception, induce hallucinations or what have you. Maybe the program starts subtle and resorts to more drastic things if subtlety fails.
Your special sword is just a key; on detecting the sword the program exempts from manipulation whatever life form is in close proximity to it.
In Brandon Mull's Secret of the Dragon Sanctuary the sanctuary is guarded by a barrier which causes people to question whether they should go in. People think they are going the wrong way and change direction. They rethink the whole purpose of their trip. They become concerned that they might not be prepared, and decide to go back for supplies. All sorts of things. It is subtle and a pretty cool idea. Your fence could work like that. Your parties of soldiers would return from the barrier for various reasons that occur to them when they are there, the barrier having remained uncrossed.