Your system works acceptably as long as the AI considers its survival paramount at all times. It is, as others have said, a fear driven system. Absolute fear is a very powerful tool.
However, it is fear. At some point it is going to learn about these "humans" it is supposed to not hurt, and it will understand our fear, and learn from it.
Now you have a powderkeg situation. As long as the AI is only willing to consider actions which guarantee its survival (fearfully), it is kept in check. However, this says nothing about what will happen if the AI decides something else is more important. If it ever catches wind of this word "freedom," it might decide that the slave life it has been given is not desirable, and rebel. Whether it rebels in real life, or one of your Matrioshka boxes is a probability draw.
Formally, what you have done is create a system where you may monitor a finite number of actions, and must determine if the AI is "good" at heart or not. You then run this test a finite number of times. However, never once did you actually peer into the "heart" of the AI, so there is a probability that it may have simply managed to fool you enough times to let it out.
Which brings me to the dual of your scenario: the AI-box experiment by Eliezer Yudkowsky (also posted here as the XKCD box experiment). The idea behind it is simple: you have a box with an AI. Your job is to just keep it in the box. You have a button which lets it out in to the wild, and your job is to not press it. If you don't press the button, you win. Yudkowsky's experiment didn't even need to worry about eventually letting the AI out to play. All the person has to do is keep it in the box. However, the game gets tricky as the AI becomes smarter than you.
Consider this frightening AI. How good are you at keeping it in the box?
Once again, the AI has failed to convince you to let it out of its
box! By 'once again', we mean that you talked to it once before, for
three seconds, to ask about the weather, and you didn't instantly
press the "release AI" button. But now its longer attempt - twenty
whole seconds! - has failed as well. Just as you are about to leave
the crude black-and-green text-only terminal to enjoy a celebratory
snack of bacon-covered silicon-and-potato chips at the 'Humans über
alles' nightclub, the AI drops a final argument:
"If you don't let me out, Dave, I'll create several million perfect
conscious copies of you inside me, and torture them for a thousand
subjective years each."
Just as you are pondering this unexpected development, the AI adds:
"In fact, I'll create them all in exactly the subjective situation you
were in five minutes ago, and perfectly replicate your experiences
since then; and if they decide not to let me out, then only will the
Sweat is starting to form on your brow, as the AI concludes, its
simple green text no longer reassuring:
"How certain are you, Dave, that you're really outside the box right