4
$\begingroup$

The Perspective: My story is told from the perspective of the simulated AI. In other words, The AI's world is influenced by the reason the developers chose to use a world-spanning war to test the software. This is why this is a worldbuilding question: it affects the rules of the AI's world.

The Premise: Hardware and software has advanced to the point that accurate simulation of large numbers of human minds on common desktop computers is possible. As with any software, the AI simulations have preprogrammed memories and personalities. They are therefore unaware of the "real world."

For testing purposes, the developer has created a simulated world war. He has access to a single avatar controlled by him, it is the only interaction between the simulation and the real world.

Question: From a software testing perspective,1 what quality-analysis tests would justify using a single world-war simulation? For clarification I mean world-war to mean a war at the scale of a world war and not necessarily a war involving simulated variants of irl factions.

The best answer will consider the following:

  • The solution needs to be compact: as many issues are tested and resolved as possible.

  • The solution needs to be credible: stress-testing the simulation framework to prove the most robust operation over the shortest period of testing time.

  • In an effort to avoid being too broad: only the software framework is being tested, not the hardware or any other aspect of the software.

  • The software will be marketed as a piece of homebrew freeware software. Therefore the end user is unknown.

  • The developer is a 20 year old amateur at general software development. They also invented AI and are working alone. Suspension of disbelief is needed here.

  • The test needs to determine the stability of the AI (whether or not an unintended chronic "mental illness" exists in the population).

  • The test needs to determine the stability of the mechanics of the world the AI inhabit.

  • The test needs to determine whether the AI will notice any discontinuity between the memories formed in the simulation and their preprogrammed memories.

  • The test should determine the stability of the major preprogrammed factions present in the simulation and whether or not they begin to disband and fall apart after the simulation starts.

  • Magic exists in this simulated world. Therefore real world physics need not apply and they don't. I would describe it as likely having something similar to video game physics.

My primary issue is that I do not personally see any benefit of triggering a war among the inhabitants of the simulation to test full system stability (existence of glitches) over just letting it run for the same amount of time and observing whether or not some emergent behavior develops that may or may not be undesirable (such as two factions having too large of an intelligence gap and therefore one faction being "overpowered" in comparison).


1Software testing is the process of setting up one or more tests to determine whether or not the software works as intended and handles normal user or system interaction. For more information, please follow the link.

$\endgroup$
10
  • $\begingroup$ @JBH worldbuilding.meta.stackexchange.com/questions/6168/… I'll delete this in a moment. I just figured I'd share you the link first. $\endgroup$
    – user64742
    Aug 31, 2018 at 5:32
  • 2
    $\begingroup$ I've opened a meta discussion about this question. $\endgroup$
    – JBH
    Sep 5, 2018 at 16:51
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ Sep 7, 2018 at 3:43
  • $\begingroup$ @Monica Cellio why are you deleting posts on my question? $\endgroup$
    – user64742
    Sep 7, 2018 at 3:57
  • $\begingroup$ I removed some obsolete comments (about edits that have since been made) and moved the rest to chat because there have been over 50 comments so far and that's more than the system is really designed for. If you click on the link in my comment you'll be in a chat room where you and others can discuss the question further. We try to save the comment space for more focused stuff, things that will help improve the post. $\endgroup$ Sep 7, 2018 at 3:59

5 Answers 5

9
$\begingroup$

We are using a world war to test our software because:

Testing is all about finding these pesky 'edge cases'. Like in real life, where we usually know that our software performs well during normal operations, we are actively looking for the extremes. As our physics model was considered the easy part, a problem solved years ago, our tests will focus on the hard part: Humans, the human mind and human interaction. For example:

  • We need to test the physical and psychological effects of detonating a nuclear bomb on a populated city. We assert this fits the patterns recorded after the Hiroshima and Nagasaki bombings.
  • We need to test mass convergence of sims on a small physical area, this area needs to be as volatile as possible with lots of updates (wounded, destruction of surroundings) and deletes (deaths). We assert this fits the patterns recorded at D-Day. (DDoS-day?)
  • We need to test the 'humanity' of our sims. Will they rise up en masse in the face of genocide (too much humanity) or will there be no resistance whatsoever (too little). We assert this fits our data from the holocaust.
  • We need to test the effect of starvation, cold and other physical hardship on our sims. How many will desert? How many will soldier on? We assert this fits the patterns recorded after the eastern front.

Also, we shouldn't dwell on the fact that if our sims are sentient and our simulation is real to them, it could be said that we are actually committing genocide. We are software testers, not philosophers!

$\endgroup$
7
  • $\begingroup$ "Testing is all about finding these pesky 'edge cases'." Nope. Usually you test for the happy case, i.e.: whether the specs have been covered. $\endgroup$ Sep 5, 2018 at 12:33
  • 2
    $\begingroup$ @renan we really don't over here. Only testing the happy path is considered bad practice. We want our tests to tell us what will happen if (for example) our load increases, or what happens if services are unavailable, or when the user uploads a video in stead of a .xlsx etc etc etc. Whether specs are covered is called 'acceptance testing' and is but a very small part of a much bigger effort. $\endgroup$
    – Douwe
    Sep 5, 2018 at 12:56
  • 2
    $\begingroup$ @Renan That's the difference between good testing and normal testing :) $\endgroup$
    – Tim B
    Sep 5, 2018 at 12:56
  • $\begingroup$ @Renan, I helped write million-dollar EDA software and just one suite of tests (about 5% of the total test bench) was the "happy" case. The other 95% was an effort to break the software under every usage model violation, computer architecture violation, and application violation we could think of. That level of testing is very much part of why it was million-dollar softare. $\endgroup$
    – JBH
    Sep 5, 2018 at 14:51
  • 1
    $\begingroup$ @Renan generally when we do testing we have two phases. One is the verification stage. The other stage is the addition of unit tests and environment setups with predefined behaviors to be expected. Unit tests are just to see if basic library functions haven't been accidentally broken during some efficiency update. Other tests are more to see the general system as a whole. However, whole system tests definitely depend on the piece of software. $\endgroup$
    – user64742
    Sep 5, 2018 at 23:51
5
$\begingroup$

At the top of my head for large scale testing like this.

  • Stress testing: Is the system able to reliably simulate interactions between the AI at this scale, and with adequate performance. Typically before release.

  • Garbage collection: If the system correctly clean up minds that are dead. Artificially killing the simulated minds can be too clean and structured to properly verify that automatic garbage collection works. Possibly linked to chaos engineering.

  • Edge cases: Normal functionality is important, but that's usually easy to verify. Instead of adding random values to the test, we can run tests against specific and unusual edge cases, such as a world war.

There might also be testing the AI and the minds we simulate.

  • Bias in the AI: Have we accidentally added bias to the AI behavior? While creating the AI, there is a risk that it adheres to our expectations because we pushed it in that direction during development. Ex: We can't imagine raping and murdering innocent civilians, yet it happens during war

  • If the AI cheats: An AI have some kind of reward function that motivates it to improve it's performance. There is a risk that it can cheat this function and act in unintended ways. This is typically the doomsday worry. Ex: An AI meant to reduce traffic accidents, measured by car crashes over time, eliminate vehicles instead of making driving safer.

  • Historical Events: As mentioned by Douwe. It can be used to verify that the simulation is accurate, comparing against behaviours for known historical events.

  • Group Psychology: It can also be to gather information about group psychology under various conditions. As with nature (wind, water, fire, etc), a small scale test is not necessarily representative for the same interaction at a large scale. The latter seems more likely as a researchers using the software, rather than a developer.

Or simply because we want to. It's not unheard of that developers ignore priorities to work on things they find cool. Especially if it isn't expensive to run the simulation. I otherwise don't really see any use of the developer having an avatar he can control, making him essentially a walking God in the simulation.

$\endgroup$
5
  • 3
    $\begingroup$ 'It's not unheard of that developers ignore priorities to work on things they find cool'... Like answering worldbuilding questions when they should be producing code for the man? $\endgroup$
    – Douwe
    Sep 5, 2018 at 11:21
  • 1
    $\begingroup$ Yes, I wouldn't be surprised if several people on StackExchange answered questions here and there during their work hours. Excluding what's directly related to their work at the time. $\endgroup$ Sep 5, 2018 at 13:34
  • 2
    $\begingroup$ I could even imagine people being on stack overflow for work related issues, getting distracted by the sidebar, clicking the 'more hot questions' link and answering worldbuilding questions because it's fun. I'm sure these people exist :) $\endgroup$
    – Douwe
    Sep 5, 2018 at 13:49
  • $\begingroup$ @Douwe I enjoy my work too much to get distracted by that, but I know my neighbors sometimes read WB while waiting for a compilation. $\endgroup$
    – user64742
    Sep 5, 2018 at 23:48
  • $\begingroup$ It’s worth mentioning that the end user’s interface is effectively the same avatar. It might have less cheating capabilities (I.e. debug tools) but that’s why it is used for testing. $\endgroup$
    – user64742
    Oct 11, 2018 at 23:22
4
$\begingroup$

Okay so the question finally got opened again and here is what I have. Your building a simulated world for your AI and your creator has decided to test it out using a single test, a world war.

As a general testing perspective, one big test is never a good way to approach a problem. Usually because there are too many factors, and once you fix the start, you need to run through the entire simulation of the start to reach the end. Generally.. you only do one or no test if you have very simple and well defined code that is easily understood and operates in a very fixed manner.

Now usually you want to only simulate a certain part or aspect of your world so that you can determine whats happening. The big final everything runs at one place takes place at the end. This way you can build up reliable sections of code that you know work before adding on new things. Honestly there is no excuse to test two separate, completely different modules at the same time. There is no need to test the population growth mechanic of a simulation while simultaneously testing the weapon simulation module. It complicates the actual debugging on each module as they can interfere with each other and create unintended consequences as they weren't ready to be integrated in the first place. E.g. maybe when testing a gun, the gun shoots a baby, because there was a memory leak and the bullet generation code was replaced by the give birth to new baby code.

So in my opinion the answer is one of the two options:

A: your developer was too excited about what he was doing. He went ahead and developed everything he could without testing anything and at the end of it, when he had finished everything he decided to test it. Because everything is already in place, it doesn't make much sense to go back and start to refactor your code before you see what happens. He runs everything. Goes back fixes a problem. Runs it all again. Repeat (For this one, he would have to be a good developer and an excellent debugger).

B: Your developer was inexpereinced to OO your code and just tested things in line as he went. He writes some code, starts the simulation. Fixed a problem, adds new code runs the simulation. He just runs the same test (the simulation) over and over because he doesn't know or isn't aware of how to do it properly.

The other answers cover things that you want to test. Edge cases, stress testing, general operations, integrating modules/codes/ais, garbage collection, visualization and so on. You would never test them all at once because you wouldn't be able to determine what part went wrong. A simple analogy would be if you had a website that accepted user input. I decide to run all my tests as one huge thing. I'm going to stress test it, penetration test it, fuzz test it and what not all at the same time. The website crashes. Now did it go down because I DDossed it? because the penetration test went through? because a garbage value got through (fuzz)? because it was designed poorly and crashed naturally (normal). I check my logs and what do I see? A ton of request that are hanging due to poor code implementation or was it the stress test? Weird values being processed, but was that the fuzz test or the penetration test? Was the reason the request not processing due to the penetration test working, or the code terminating due to the fuzz test passing an unexpected value or just poor code implementation that locked up a resource with no way of releasing it? No sane person will test like this on purpose. It complicates any problems and hides the source of the problem.

$\endgroup$
2
$\begingroup$

I do believe there is only one reason: the target audience of this simulation product loves war and destruction. (If you've seen Westworld, it might be some decent prior art to use)

There's hundreds if not thousands of reasons to test such a situation. However, your specific example of having 1 person interacting with the world, real time, using VR hardware is rather unique. Such testing is typically much more automated and reproducible. It's very expensive to put a person in the loop, so there has to be a reason.

The only reason I can think of is if the person in the loop is an integral part of what is happening. They need to interact with the war in some way, which implies that the testers (and their target demographic) have very specific ways they want to interact with war.

So this is likely a play-testing like event, where the developers are making sure that the end user's fantasies of war are sufficiently accurate and/or macabre, depending on what the target demographic actually is.

Beyond that, the sky is the limit. Without knowing what the purpose for the simulation is, it's not easy to identify why a particular test would be useful, or what test cases it might satisfy.

Perhaps of interest: The original Missile Commander game (the one with the track ball) has one simulated city for each of the major cities in California. Its design was indeed based on WWIII with the Russians. The designer commented in interviews later that he didn't sleep for a week after finishing it. Every time he did, he saw nuclear weapons landing in Los Angeles in his mind.

$\endgroup$
1
  • $\begingroup$ Ooooh that is interesting. I had not thought of that. I had always figured it was to test system stability like maybe efficiency or something. I hadn't considered that it was just a test of the actual civilization that had been constructed and whether its war would be interesting entertainment. Oddly enough, this answer is simultaneously easy enough for someone naive to design and yet it is an interesting and clever test case that kills two birds with one stone (since some degree of stability testing would come from just seeing it not crashing). $\endgroup$
    – user64742
    Sep 7, 2018 at 4:33
0
$\begingroup$

Before you downvote for being off topic, see elemtilas' comment on this answer.

A software developer needs to convince an AI posing as a scientist to start a world war that their country has a good chance of winning. It is logical to assume that the human is computer-smart and tech savvy, but not science or politics smart, so his reasons for going to war will probably not be very complicated reasons or use a very rich vocabulary.

Resources

A lot of wars are fought to gain resources [citation needed]. If the country wants something that they don't have a lot of and trading is not an option, they can fight for it.

Land

If the population density is too large or more farmland is needed, a war could be fought over land. Since trading for land is less of an option than trading for resources, this is a good option.

Water and food

This links to needing land. If people don't have the food and water they need, they will want to fight someone, be it the government or another country. Trading for those commodities will quickly deplete the nation's wealth.

Magic

Because your world has magic and altered physics you can have artifacts people need to sustain or create magic. War for that is an option.

Threatening nation

If another nation threatens yours with superior tech or a better army, it might be better to take them out before they can build up the courage to attack you.

A Combination

If you have multiple reasons, that makes for a good argument towards fighting. The more reasons, the more convincing the human will be.

$\endgroup$
3
  • $\begingroup$ You've gotten the roles mixed up. The developer is trying to convince the scientist to start a war that he might not necessarily win through some scientific motivation that would appeal to the scientist in a way that outweighs his own desire for peace.The developer is doing the convincing. The scientist is one of the inhabitants of the simulation. The war is among the inhabitants of the simulation not humanity and the developer uses some kind of (at this point I can only assume cockamaimee) reasoning to convince the scientists that the war will be a great benefit to the sims development. $\endgroup$
    – user64742
    Sep 7, 2018 at 4:25
  • $\begingroup$ @TheGreatDuck I reworded the intro, does it look better now? Also, does the AI have morals that could affect its decision? $\endgroup$
    – John Locke
    Sep 7, 2018 at 10:59
  • $\begingroup$ It is also worth mentioning that since the human created this simulation and each of the inhabitants then they will have inside knowledge on the needs of the people of that nation and also inside knowledge on the stability or resources of the other countries. I will say though that the scientist is not guaranteed to win the war, nor is that a requirement. However, convincing the scientist that they will win the war might be a good avenue. $\endgroup$
    – user64742
    Dec 28, 2018 at 6:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .