NOTE
It has been suggested that I change the word 'slave' to something else, perhaps 'servant'? My intention is to describe a person who is required to work for someone else with no pay. There is no desire for inhumane behaviour to be endemic - undoubtedly some people will want to take advantage of the system more than others.
I reserve the right to change the word if necessary. It won't change the nature of the question.
Background
In this medieval society there is a class of slaves. Let's call them S.
The S are servants of the R and the R in turn are servants of the Q.
The S are masters of the T and the T are masters of the U.
Thus every group known about by the S is both enslaved to one group and master to another in a hierarchy.
The catch
What nobody knows about is that this is a ploy by the Grand Wizard. He has created a circular city of huge proportions and the masters/slaves are distributed around it. They are not allowed to travel far and so they don't realise that the system is actually circular. Thus the A are the servants of the Z. This means that every single group is both slave and master.
There are no chairs in the following picture
Each person is supporting the one in front. If you were to put an opaque cylinder in the middle and tell everyone there was a chair supporting 'the last person', they could easily all believe it.
Question
I'm certain that some people would find personal loopholes, e.g. working lazily for a weak master while being simultaneously a tyrant to those below them.
However apart from this unevenness, what, if anything, could stop this circular system of master/slave from working?
Notes
It is not allowed for a servant given a task by a master to pass that task on to one of their own slaves. They must do it themselves and get their own slaves to do their tasks for them so that they can fulfil their masters' wishes.
People must perform their own duties as well as serving others. Thus they can be accountants and gardeners. Some can also be craftspeople or artists in their spare time and make extra money. They pay taxes to the Grand Wizard.
Some people having been objecting in comments that you can't serve someone else and have another occupation. Yes you can!! You have a slave to help you. Look at it this way. In modern society most people who work full time also have to come home and do their own laundry, vacuuming, tidying, paperwork etc. Instead you come home from work but do your master's laundry, vacuuming, tidying, paperwork etc. (or whatever they prefer you to do). Thus no-one necessarily works more in total than an average person would. Of course some people will exploit the system but that's true of any system.
Example
My master tells me to polish his boots. I must do it myself. Therefore I instruct my slave to polish my boots (or iron my shirt or whatever), so that I have time to polish my master's boots. This continues around the circle. Note that each of us also has their day job.