From a software testing perspective, what quality-analysis tests would justify using a single world-war simulation?

Question

The Perspective: My story is told from the perspective of the simulated AI. In other words, The AI's world is influenced by the reason the developers chose to use a world-spanning war to test the software. This is why this is a worldbuilding question: it affects the rules of the AI's world.

The Premise: Hardware and software has advanced to the point that accurate simulation of large numbers of human minds on common desktop computers is possible. As with any software, the AI simulations have preprogrammed memories and personalities. They are therefore unaware of the "real world."

For testing purposes, the developer has created a simulated world war. He has access to a single avatar controlled by him, it is the only interaction between the simulation and the real world.

Question: From a software testing perspective,¹ what quality-analysis tests would justify using a single world-war simulation? For clarification I mean world-war to mean a war at the scale of a world war and not necessarily a war involving simulated variants of irl factions.

The best answer will consider the following:

• The solution needs to be compact: as many issues are tested and resolved as possible.

• The solution needs to be credible: stress-testing the simulation framework to prove the most robust operation over the shortest period of testing time.

• In an effort to avoid being too broad: only the software framework is being tested, not the hardware or any other aspect of the software.

• The software will be marketed as a piece of homebrew freeware software. Therefore the end user is unknown.

• The developer is a 20 year old amateur at general software development. They also invented AI and are working alone. Suspension of disbelief is needed here.

• The test needs to determine the stability of the AI (whether or not an unintended chronic "mental illness" exists in the population).

• The test needs to determine the stability of the mechanics of the world the AI inhabit.

• The test needs to determine whether the AI will notice any discontinuity between the memories formed in the simulation and their preprogrammed memories.

• The test should determine the stability of the major preprogrammed factions present in the simulation and whether or not they begin to disband and fall apart after the simulation starts.

• Magic exists in this simulated world. Therefore real world physics need not apply and they don't. I would describe it as likely having something similar to video game physics.

My primary issue is that I do not personally see any benefit of triggering a war among the inhabitants of the simulation to test full system stability (existence of glitches) over just letting it run for the same amount of time and observing whether or not some emergent behavior develops that may or may not be undesirable (such as two factions having too large of an intelligence gap and therefore one faction being "overpowered" in comparison).

---

_{¹ Software testing is the process of setting up one or more tests to determine whether or not the software works as intended and handles normal user or system interaction. For more information, please follow the link.}

Answer 1

We are using a world war to test our software because:

Testing is all about finding these pesky 'edge cases'. Like in real life, where we usually know that our software performs well during normal operations, we are actively looking for the extremes. As our physics model was considered the easy part, a problem solved years ago, our tests will focus on the hard part: Humans, the human mind and human interaction. For example:

• We need to test the physical and psychological effects of detonating a nuclear bomb on a populated city. We assert this fits the patterns recorded after the Hiroshima and Nagasaki bombings.
• We need to test mass convergence of sims on a small physical area, this area needs to be as volatile as possible with lots of updates (wounded, destruction of surroundings) and deletes (deaths). We assert this fits the patterns recorded at D-Day. (DDoS-day?)
• We need to test the 'humanity' of our sims. Will they rise up en masse in the face of genocide (too much humanity) or will there be no resistance whatsoever (too little). We assert this fits our data from the holocaust.
• We need to test the effect of starvation, cold and other physical hardship on our sims. How many will desert? How many will soldier on? We assert this fits the patterns recorded after the eastern front.

Also, we shouldn't dwell on the fact that if our sims are sentient and our simulation is real to them, it could be said that we are actually committing genocide. We are software testers, not philosophers!

Answer 2

At the top of my head for large scale testing like this.

• Stress testing: Is the system able to reliably simulate interactions between the AI at this scale, and with adequate performance. Typically before release.

• Garbage collection: If the system correctly clean up minds that are dead. Artificially killing the simulated minds can be too clean and structured to properly verify that automatic garbage collection works. Possibly linked to chaos engineering.

• Edge cases: Normal functionality is important, but that's usually easy to verify. Instead of adding random values to the test, we can run tests against specific and unusual edge cases, such as a world war.

There might also be testing the AI and the minds we simulate.

• Bias in the AI: Have we accidentally added bias to the AI behavior? While creating the AI, there is a risk that it adheres to our expectations because we pushed it in that direction during development. Ex: We can't imagine raping and murdering innocent civilians, yet it happens during war

• If the AI cheats: An AI have some kind of reward function that motivates it to improve it's performance. There is a risk that it can cheat this function and act in unintended ways. This is typically the doomsday worry. Ex: An AI meant to reduce traffic accidents, measured by car crashes over time, eliminate vehicles instead of making driving safer.

• Historical Events: As mentioned by Douwe. It can be used to verify that the simulation is accurate, comparing against behaviours for known historical events.

• Group Psychology: It can also be to gather information about group psychology under various conditions. As with nature (wind, water, fire, etc), a small scale test is not necessarily representative for the same interaction at a large scale. The latter seems more likely as a researchers using the software, rather than a developer.

Or simply because we want to. It's not unheard of that developers ignore priorities to work on things they find cool. Especially if it isn't expensive to run the simulation. I otherwise don't really see any use of the developer having an avatar he can control, making him essentially a walking God in the simulation.

Answer 3

Okay so the question finally got opened again and here is what I have. Your building a simulated world for your AI and your creator has decided to test it out using a single test, a world war.

As a general testing perspective, one big test is never a good way to approach a problem. Usually because there are too many factors, and once you fix the start, you need to run through the entire simulation of the start to reach the end. Generally.. you only do one or no test if you have very simple and well defined code that is easily understood and operates in a very fixed manner.

Now usually you want to only simulate a certain part or aspect of your world so that you can determine whats happening. The big final everything runs at one place takes place at the end. This way you can build up reliable sections of code that you know work before adding on new things. Honestly there is no excuse to test two separate, completely different modules at the same time. There is no need to test the population growth mechanic of a simulation while simultaneously testing the weapon simulation module. It complicates the actual debugging on each module as they can interfere with each other and create unintended consequences as they weren't ready to be integrated in the first place. E.g. maybe when testing a gun, the gun shoots a baby, because there was a memory leak and the bullet generation code was replaced by the give birth to new baby code.

So in my opinion the answer is one of the two options:

A: your developer was too excited about what he was doing. He went ahead and developed everything he could without testing anything and at the end of it, when he had finished everything he decided to test it. Because everything is already in place, it doesn't make much sense to go back and start to refactor your code before you see what happens. He runs everything. Goes back fixes a problem. Runs it all again. Repeat (For this one, he would have to be a good developer and an excellent debugger).

B: Your developer was inexpereinced to OO your code and just tested things in line as he went. He writes some code, starts the simulation. Fixed a problem, adds new code runs the simulation. He just runs the same test (the simulation) over and over because he doesn't know or isn't aware of how to do it properly.

The other answers cover things that you want to test. Edge cases, stress testing, general operations, integrating modules/codes/ais, garbage collection, visualization and so on. You would never test them all at once because you wouldn't be able to determine what part went wrong. A simple analogy would be if you had a website that accepted user input. I decide to run all my tests as one huge thing. I'm going to stress test it, penetration test it, fuzz test it and what not all at the same time. The website crashes. Now did it go down because I DDossed it? because the penetration test went through? because a garbage value got through (fuzz)? because it was designed poorly and crashed naturally (normal). I check my logs and what do I see? A ton of request that are hanging due to poor code implementation or was it the stress test? Weird values being processed, but was that the fuzz test or the penetration test? Was the reason the request not processing due to the penetration test working, or the code terminating due to the fuzz test passing an unexpected value or just poor code implementation that locked up a resource with no way of releasing it? No sane person will test like this on purpose. It complicates any problems and hides the source of the problem.

Answer 4

I do believe there is only one reason: the target audience of this simulation product loves war and destruction. (If you've seen Westworld, it might be some decent prior art to use)

There's hundreds if not thousands of reasons to test such a situation. However, your specific example of having 1 person interacting with the world, real time, using VR hardware is rather unique. Such testing is typically much more automated and reproducible. It's very expensive to put a person in the loop, so there has to be a reason.

The only reason I can think of is if the person in the loop is an integral part of what is happening. They need to interact with the war in some way, which implies that the testers (and their target demographic) have very specific ways they want to interact with war.

So this is likely a play-testing like event, where the developers are making sure that the end user's fantasies of war are sufficiently accurate and/or macabre, depending on what the target demographic actually is.

Beyond that, the sky is the limit. Without knowing what the purpose for the simulation is, it's not easy to identify why a particular test would be useful, or what test cases it might satisfy.

Perhaps of interest: The original Missile Commander game (the one with the track ball) has one simulated city for each of the major cities in California. Its design was indeed based on WWIII with the Russians. The designer commented in interviews later that he didn't sleep for a week after finishing it. Every time he did, he saw nuclear weapons landing in Los Angeles in his mind.

Answer 5

Before you downvote for being off topic, see elemtilas' comment on this answer.

A software developer needs to convince an AI posing as a scientist to start a world war that their country has a good chance of winning. It is logical to assume that the human is computer-smart and tech savvy, but not science or politics smart, so his reasons for going to war will probably not be very complicated reasons or use a very rich vocabulary.

Resources

A lot of wars are fought to gain resources [citation needed]. If the country wants something that they don't have a lot of and trading is not an option, they can fight for it.

Land

If the population density is too large or more farmland is needed, a war could be fought over land. Since trading for land is less of an option than trading for resources, this is a good option.

Water and food

This links to needing land. If people don't have the food and water they need, they will want to fight someone, be it the government or another country. Trading for those commodities will quickly deplete the nation's wealth.

Magic

Because your world has magic and altered physics you can have artifacts people need to sustain or create magic. War for that is an option.

Threatening nation

If another nation threatens yours with superior tech or a better army, it might be better to take them out before they can build up the courage to attack you.

A Combination

If you have multiple reasons, that makes for a good argument towards fighting. The more reasons, the more convincing the human will be.