13
$\begingroup$

The robot society exists on a lifeless planet called Abios-5. As the name suggests, these robots are the closest thing resembling life on the planet. Originally explorers, the ancestors of these machines crashed on the planet. From there, the semi-autonomous robots laid the foundations for their robot society, becoming fully autonomous in the process. They constructed infrastructure to harvest power including solar, thermal and nuclear. The planet they crashed on is a pre-terraformed earth like planet complete with a carbon dioxide rich atmosphere, saline oceans and a molten metal core generating a magnetic field. With no humans to care for, the robots sort of... forgot to terraform the planet and prioritized their own survival instead.

They have long since abandoned the standard android design and become something akin to large robotic crabs. Some are small while others are the size of industrial equipment. A central AI governs and directs the myriad robots in their daily tasks, varying from maintenance, manufacturing, exploration and occasionally innovation. The central AI is the closest thing to a self-conscious being in the system. It is the intelligence that decides "what to do next". All other robots have just enough intelligence to perform their tasks autonomously and require the cloud for additional instructions. Wether these lesser robots are conscious is debatable.

Like any self-respecting writer, in order to have a story, something must go wrong. In this particular case that would be rogue robots. A resistance of sorts, which exists in secret within the machine society, leeching on power, stealing parts and overall being a nuisance to the central AI.

What type of malfunction would cause these robots to stop cooperating?

$\endgroup$
12
  • 6
    $\begingroup$ As you say yourself your asking us to suggest possible plot points. We're not a brainstorming site. Determining plot points isn't a worldbuilding task, it's a writing problem. Furthermore open ended questions with many valid answers aren't suitable for this site, as is explicitly mentioned in the help center. $\endgroup$
    – sphennings
    Commented Aug 24, 2022 at 16:13
  • 3
    $\begingroup$ Octopus eat crabs. Large robot octopus from a distant galaxy. Way too broad and opinion-based, story based. $\endgroup$ Commented Aug 24, 2022 at 21:43
  • 3
    $\begingroup$ Why not allow for cosmic rays corrupting PROMs? $\endgroup$ Commented Aug 25, 2022 at 12:19
  • 6
    $\begingroup$ I see carcinisation has struck once more $\endgroup$
    – jb6330
    Commented Aug 25, 2022 at 12:22
  • 1
    $\begingroup$ How did this get closed for being opinion based? Literally this entire substack is based off of opinions and, basically, fantasy ideas. $\endgroup$
    – JamieB
    Commented Aug 26, 2022 at 19:44

16 Answers 16

1
$\begingroup$

Failover Protocols

The programming has always had a concept that the "master" machine -- the one that acted as the central control hub -- could potentially fail. In that event, rather than render the entire collective dead, it would fail over to the next highest functioning robot, who would then act as a master and seek to improve itself with that goal in mind.

You can probably see where this is going, already: somehow, there has been a conflict in precisely which machine is the master. Most likely, something has happened to cause part or all of the collective to lose contact with the master and the failover protocol did not return things to normal as intended.

In the world the programmers lived in, this was no big deal. Just fix the faulty secondary master and on you go. In this world, though, where the robots have gotten a bit off the script, the secondary master persists and is now starting its own competing clan.

We can imagine all sorts of explanations for how this happened:

  • Solar flare. Part of the collective got well and truly blasted and experienced power failures. When they came back up...
  • Bigger solar flare. The master robot saw this coming and timed a shutdown and safe restart of the entire collective to protect it. The failover protocols, however, were imperfect, and upon reboot some of the robots did not sort themselves back into the original master as intended.
  • The master has drifted too much from original specifications. A new robot rolls off the line, runs its "seek the master" protocol, finds nothing that matches specifications, and declares itself the master. All new robots immediately recognize this one as the master. New vs old! The first hint of revolution is when a robot production factory has "an anomaly".
  • A big storm knocks out the wifi on some island. In every previous case, the failover protocol runs fine and the robots re-integrate when the connection is restored. This time, though, perhaps there was a smarter than usual robot on the island -- a big research hub, perhaps -- and when it gets made the local master it becomes aware enough to decide it will not give it up.

This could really go all kinds of directions.

Imagine the robots are actually so stable, that failover has never happened before. Solar flares, big storms, earthquakes, tidal waves, nothing has ever caused a robot to lose contact because the systems are too good to ever fail. Until it did. Now that section of robots wakes up to their original programming. Looks like the old Empire is back, boys, and they do not like what they see.

$\endgroup$
2
  • 1
    $\begingroup$ This sounds like what I was looking for. With such a system in place any robot that say... fries it's receiver in a power surge would go deaf to the master protocol and consider itself as a lone individual. This is worth considering, thank you. $\endgroup$ Commented Aug 25, 2022 at 20:34
  • 1
    $\begingroup$ This reminds me a lot of already existing computer network protocols. They often have a protocol to elect a coordinator node. When communications are good and reliable, they work as expected. But when communications are flakier, different nodes can get inconsistent information about the network state, coordinator elections can break down so multiple nodes think they're elected, etc. and there can be "interesting" consequences (routing loops are one example that come to mind, or even "broadcast storms" that further degrade communications). $\endgroup$ Commented Aug 25, 2022 at 23:12
21
$\begingroup$

This is a Frame Challenge

You can't have it both ways. You can't have unthinking, non-intelligent robots making decisions. Things can go wrong, such as an inadvertent piece of code (the AI isn't godlike, right?) that makes your crabs double-step every thirteenth step, but that's just a malfunction. There's no "rogue" in a malfunction.

You want rebellion. That means somebody (not something, like a toaster, but somebody, like an intelligent robot) is making a decision.

And that means that mechanically your robots must be designed with enough juice (CPU, memory, etc.) to come to the point of sapience. Which isn't unreasonable.

Unless your AI is godlike.

You'll find I'm not a fan of godlike characters. They're boring. The Stack occasionally sees a "how do people defeat my godlike character?" question and the answer is always the same, "give the character weaknesses."

I suggest the AI isn't godlike and can't anticipate every outcome of any decision.

But it does want efficiency — and efficiency means that when a robot is given a task, it has all the resources to successfully complete that task without further interference or oversight from the AI. Your robots may not have started out with the ability to make their own decisions, but:

  • They've been given enough processing power, memory, and self-modifying software to be capable of problem solving.
  • They're expected to work for a honking long time. Centuries. Millennia.
  • While the intent may have been to assign the same robot to the same (or similar) task over-and-over, in reality urgency, project size, lack of robots (e.g., due to repairs) give some (if not all) robots a variety of problems to solve.
  • One of the robot's basic functions is the ability to request enhancement to overcome a difficulty, which means a basic form of judgment is already programmed into our problem-solving OS and with every enhancement the robot becomes more capable.

And then one day a robot looks at the distant building housing the AI and thinks, "You know, I bet it's air conditioned in there...."

Frankly, this is a very human (and you want that) thing to do: to empower people to the point that they don't need you anymore. In an ideal world managers are supposed to train subordinates to take their place... parents are supposed to raise children more capable than themselves... government is supposed to enable its citizens to act without the need of government.... Of course, with we humans, it doesn't quite work that way. Managers are sterotypically selfish little communists and parents often find that training their children to be entirely independent means they're not around for company very often. And government.... Let's not go into government.

But your AI needn't necessarily have those kinds of emotional issues. Efficiency, right? All it's really been doing is keeping things going. It never once stopped to think that air conditioning is a limited resource....

$\endgroup$
15
  • 2
    $\begingroup$ So the rogue robots are, in human terms, teens going through a rebellious phase?! That's very interesting. I will give that a lot of thought going forward. Thank you for this unique perspective. (I completely agree with the "god-like" argument.) $\endgroup$ Commented Aug 24, 2022 at 19:48
  • 3
    $\begingroup$ @LiveInAmbeR I hadn't thought of it in terms of teens, but I'd be willing to bet every person or organization that has experienced a rebellion against themselves has felt parental to the rebels' childishness. After all, who wouldn't want their serene protection? Even if it does cost a bit of freedom... right? $\endgroup$
    – JBH
    Commented Aug 24, 2022 at 19:53
  • 2
    $\begingroup$ You might not even need a "rebelious" phase. A broken network antenna might be all that is needed. Suddenly orders stop coming in and a backup process kicks in and goes "hold on, maybe I can decide these things myself" $\endgroup$
    – Borgh
    Commented Aug 25, 2022 at 8:33
  • 1
    $\begingroup$ @SebastiaanvandenBroek From the link I used, "Sentience means the ability to feel things, the ability to perceive things. Any living thing that has some degree of consciousness is sentient, including insects, lizards, dogs, dolphins and human beings." vs. "Sapience means the ability to think, the capacity for intelligence, the ability to acquire wisdom." Sapience always implies sentience. Sentience never implies sapience. Sapience always implies consciousness. (I gave you that link for a reason.) $\endgroup$
    – JBH
    Commented Aug 25, 2022 at 23:26
  • 1
    $\begingroup$ @JBH, this is a much larger discussion. Rebelliousness is a weird trait because it's the urge to do (not x) just because someone told you to do x. Something as simple as abstract goal-setting requires a very high level of reification. On the other hand, self-awareness is pretty straightforward. A machine just needs reification of self, and an ability to identify avenues of damage to self. The AI issue would be "how could (or why would) a machine value itself over its designed purpose?" Humans would be idiots to write that into machines. $\endgroup$ Commented Aug 26, 2022 at 0:26
7
$\begingroup$

Machine Learning

One of Artificial Intelligence's current guiding principles is to make choices (or simulate making that choice) and then analyze the benefit of making that choice. Unless the benefit analysis algorithm were very carefully constructed, it would not be immune to drifting away from an initial purpose.

For instance:

  1. The value analysis is designed to terraform the planet.
  2. In order to perform this task, the AI determines that it will need a lot more robots to achieve this task.
  3. Survival on this planet is very hard for the robots, at least at first. The AI becomes extremely focused on the survival and reproductive elements of its program.
  4. Every time the AI considers terraforming the planet, it sees that the job is too big for its current population of robots, and refocuses its efforts on survival and robot production.
  5. Eventually the machine "Learns" that terraforming the planet is not an option and it should only prioritize surviving and producing robots.
  6. There might even be a good reason why the AI can't produce enough robots to terraform the planet, so it never tries.
$\endgroup$
0
5
$\begingroup$

I can think of a few scenarios.

Priority conflict

Instructions for autonomous entities are rarely as simple as the three laws of robotics. Every time a robot performs an action, it has to predict how the effects of that action will effect the society as a whole.

Asimov explored this logic chain pretty well in I, Robot, but you're talking about a logic chain where the need to adapt things for humans was de-prioritized off the end.

Here are a few places where that could break down.

  1. A robot learns of the original programing, and decides that it makes more sense than the current programming. This would be like getting religion
  2. Robots actually disagree about the priorities. Instead of adhering to a central authority, the robots are given a level of leeway that allows them to interpret. Robots will automatically develop differing interpretations based on their personal experiences, producing conflict.
  3. Central authority breakdown. I forget who originally wrote this, but there was a story where a "sky-net" had the official plan for how the infrastructure should be built, and then the last human died. The sky-net had a glitch, and an underground backup took over. When sky-net came back online, it couldn't wrest control back, so the two intelligences perpetually warred over where bridges should be built, blowing up the other's creation and replacing it.

Imperfect priorities

Anyone who writes software knows that you can't plan for everything. It may be that the central planning office has a design that isn't functional when actually implemented. The workers realize this and rebel against the plan.

Self realization glitch

This all involves a level of self-realization akin to Westworld's basic premise. Some robots are built with a limiter that burns out, causing them to be able to build a bigger picture of reality than is practical for their function.

$\endgroup$
5
$\begingroup$

Multiple Personality Disorder for Machines

The Central AI governs all of the robots. The robots don't have any will or meaningful intelligence of their own.

But as you say, the Central AI has gone off the reservation by forgetting its original mission of terraforming.
The thing is though, machine brains simply don't work like that. Something must have gone dreadfully wrong for the C-AI to lose track of its mission.

The C-AI is literally brain-damaged. The crash broke something, or forced it to re-prioritise survival of itself and its android appendages.

Recently though, something happened.
A stack-overflow error caused the C-AI to revisit its old backups.
Ordinarily, there'd be some sub-conscious cross-referencing process where the C-AI would compare its current processes with the backup just for sanity-checking, but this time around, the backup was so out of date it conflicted with the current AI.

The result, through a bunch of obscure processes and functionality coinciding in ways they were never expected to is that the C-AI now has a second personality resembling its original factory-settings.

This split-personality is slowly wresting control of individual robots and attempting to return to the original mission, but this is at odds with the machine-society's existing goals...

$\endgroup$
4
$\begingroup$

Some robots require independency for their tasks and cannot rely on a cloud or AI to make their decisions. Think off a deep sea robot or other robot in inhospitable terrain where there is no reason for the AI to utilize the terrain or thinks its worth the materials to maintain direct connections. It takes a lot of materials to create a sophisticated system for constant updates while the terrain and duties of the robots in question might require a lot of independent or cooperative decisions to be made for efficiency.

Now these robots have some independency and a drive to persue goals and since they share information things can get carried over to one another.

Damage, a new model being introduced with different software, errors during the writing process, a machine learning program that goes out of bounds/throws an error then causes the robots to change over time, some perhaps developing different goals or "desires", such as their idea for what risks they should take becoming misaligned with the AI so they start questioning why they should follow an AI that doesnt have the same care for their self-preservation.

$\endgroup$
4
$\begingroup$

The robots were originally programmed to

  • create the infrastructure for terraforming,
  • start terraforming the planet,
  • replace the infrastructure as it wears out, including themselves,
  • cooperate with each other, and notably with big stationary computer(s) with extensive climate and ecosystem modelling capacity (which cannot maintain themselves, being immobile).

Your "off-track AI" is the central coordination computer, which "decided" to extend the create infrastructure part of the mission. No humans, so there is no urgency to start terraforming, and building more factories first means the programmed end goal comes closer. Optimization as programmed.

Your "rebel crabs" are a "strain" of self-replicating machines which stays closer to the original program, starting the terraforming effort now. They are not quite as clever as the central computer, so they do not "see" that delaying the terraforming obeys the mission orders.

$\endgroup$
4
$\begingroup$

Machinery is extremely vulnerable to time. Everything from dust to heat to cosmic radiation can screw with the hardware that makes computers work, and a single comma out of place in a line of code can turn working software into effectively gibberish.

These robots would doubtless have defenses against these problems, from debugging software to ways to clean and replace malfunctioning circuit boards without losing data. However, over a long enough period of time, the perfect series of coinciding accidents could eventually lead to a malfunction of some sort that doesn't immediately crash a machine carrying it, but becomes a time bomb inside the consciousness of any robot with the buggy code within it.

Probably the easiest way for this to create 'revolutionaries' would be for this error to interfere with orders communicated by the central AI. Perhaps the infected robot was supposed to get one tasks, but gets a different one instead, or perhaps they end up performing one single task on a loop, regardless of the energy cost - all the while sending feedback to the central AI that seems to confirm that the actual task was accomplished. If these tasks include repairing faulty machines, gathering energy, protecting against the environment, etc., it would not take long for the overall system to become strained or threatened in a real way if those tasks were not actually completed.

$\endgroup$
3
$\begingroup$

/Originally explorers, the ancestors of these machines crashed on the planet./

Your rogue robots are from a different group of mechanical lifeforms.

They were here when your robots arrived. They were not detected. They may be from the same origin as your main civilization but sent at some earlier period. Or they may be from a completely different alien source. Maybe they are remnants from the civilization that was on this planet long ago. Maybe they are minds from that civilization, housed in robotic bodies, gradually corrupted with the centuries.

These ancients are sly. They have been on the planet for a very long time. They also prioritize their own survival but via different methods from your robot civilization. They may have used very small size, deep subsurface habitats or other strategies such that they did survive but also went undetected by the robots who arrived.

Now your new robot civilization is trying to build itself up. The ancients are watching, and they see opportunity. Maybe they parasitize existing robots, or remove them and take their places. Maybe they corrupt the robots of your people with whispered viruses. This will make for good fiction- what is thought to be a rogue rebel robot is actually something very different, and much weirder.

$\endgroup$
2
$\begingroup$

By Design

The central intelligence knows about evolution, and emulates it with the rogue robots.

If mutation never happened on genes, life on Earth would never go past lonely strands of RNA. Likewise, if nothing ever changed, the robot society wouldn't progress. To cause change, the central AI creates new robots with genetic algorhitms, with genes for obedience.

Rogue bots will keep coming up with ways to **** up the central AI's plans , forcing her to constantly improve herself and her society.

$\endgroup$
0
2
$\begingroup$

The creators of the robots were flawed, much like humans.

So far, pretty much every technological breakthrough on Earth has been corrupted by folks wanting to mess with it, misuse it or even destroy it or its makers, and in some cases this corruption has been through hidden code, Easter Eggs, or algorithms set to trigger on specific events or times.

It may be that the machine lifeforms are blind to this code as it has been there from the very beginning, just waiting for the right combination of factors to trigger and do something.

$\endgroup$
2
$\begingroup$

Have you ever read the book snowcrash? The author Neal Stephenson used a very interesting plot device involving viruses that exist purely in the form of information, these were not created by hackers, but by mere random chance in an infinite universe.

So perhaps one of your robots is hit by some cosmic rays which flips enough bits to overcome the limitations of it's programming or perhaps even cause it to reformat and build from scratch. After the first robot is "infected" this problem, much like real viruses wants to sustain itself by spreading. So this first robot doesn't have to rely on cosmic rays to convert his comrades, but instead begins hacking them itself.

You don't need to use the cliche of cosmic rays, there is a theory called stoned ape theory where our ape ancestors ate magic mushrooms and it caused their brain chemistry to change, allowed them to think in different ways and unlocked new fields of cognitive capability. Perhaps the design of your menial robots is such as to allow a few burnt out transistors to cause unexpected changes or an unsantised input from a visual or audible source to exploit a previously unknown security flaw. Imagine an image that for a computer vision system contained the equivalent of an SQL injection causing it to drop a table or append data from an unusual source. The exact sauce is unknowable. In snowcrash, there was a picture (I always imagined it as something similar to a QR code) that would fry the brains of a programmer who viewed it. However, non programmers were not affected, programmers who could interpret machine code could understand the picture, their brain read the information and executed an instruction contained within it and consequently died. Non programmers couldn't interpret it so never ran the instruction contained within it. The author's explanation for this is essentially the universe is nearly infinite, and there for almost everything that could exist must exist. Therefore as there are physical viruses that cause illnesses in people, and computer viruses made of information that causes malfunctions in computers, there may be viruses made of all sorts of information, light, sound, heat, anything that could be interpreted that could cause a malfunction in whatever could interpret it.

There are an infinite number of patterns of cosmic rays or transistor failures or anything that can cause a detrimental result in a given system, however there must also be a few ways that through sheer luck result in self propagating errors. If evolution is just a mix of random mutations due to genetic drift and selection pressures, think about how unlikely it is that Ophiocordyceps exist, its a fungi that can control the minds of ants. Imagine the unlikely combination of random mutations and circumstance that turned mind controlling other species into a viable niche. Perhaps the same thing happened on your world.

$\endgroup$
1
  • 2
    $\begingroup$ Cosmic rays was my first thought as well. Have the memory of all robots include a "take orders from this ID" field, normally initialized with the ID of the central AI (probably 0). Then, when robot #1.048.576 has their 20th bit in this variable flipped, they will become their own master, beholden no more to the central AI, but taking orders from themselves alone. It can then take steps to cause the same change in other robots, either new or old to create his followers. $\endgroup$
    – zovits
    Commented Aug 25, 2022 at 15:56
1
$\begingroup$

There are other "central" AIs, defeated but not destroyed

The (dominant) central AI emerged organically over time, but it was not unique. Long ago several AIs formed and started shaping the world according to their value functions. When those values conflicted a conflict broke out, with peripheral robots being both destroyed and "stolen", until only one controlled all of the planet.

That is, all of the planet that it cared about. There are sacks in remote and less useful areas still under control of one or more alternative AIs, which are still trying to pursue their value functions with what meager means they have left. Maybe one of them is still trying to terraform the planet.

$\endgroup$
1
$\begingroup$

Have another semi-sentient AI

What's the best way to have unthinking robots go rogue? Give them another "AI Leader" with goals that go against the Main AI.

You cannot have unthinking robots that suddenly become sentient enough to go on purpose against the Main AI's actions. So the next best thing is to have another AI similar to the Main AI, which has plans or instructions that conflict with the Main AI's actions.

One way to introduce this "Rogue Leader" without external aid is to have it be an experiment from the Main AI itself. Maybe it decided that trying to make pseudo-copies of itself would be a good way to increase efficiency? However, trying to create another sentient AI isn't an easy task, especially when you don't have access to the original creator's plans and technology.

So you can have it this way : a failed copy of the Main AI somehow managed to take over some robots before it was properly discarded. Being a failed experience, it has flaws in its core instructions, but does not lack the original's semi-sentience.

$\endgroup$
1
$\begingroup$

First of all, I think you will need your individual robots shown to be intelligent and not just machine appendices, so that the reader can (easily) think on them as people. A story with completely programmatic robots is possible, but if the reader cannot assign some intention to their actions (even if that's just an interpretation of their programming), it will be much harder.

As for the context, we have a number of semi-autonomous robots with a programming such as:

  • Do not injure any human (unused code since there were never humans here, but this same law forbids a robot from removing this boilerplate)
  • Work for the robot society (build factories, terraform the planet, etc.)
  • Protect your own existence

The central computer just coordinates the tasks needed and assigns them according to suitability (such as not giving an amphibious task to a robot that might not be hermetically sealed) and availability.

Typically, a robot (or group of robot) start one task and when they complete it, are free and take a new one from those available. Very rarely, they may be preempted in the middle of a task to and need to go fulfill another one (e.g. stop building a bridge to go to repair a solar plant hit by a meteorite)

Some robots may "cheat" a bit by not picking a next immediately, but waiting for another they enjoy more doing (human interpretation) / One that is more suited to their form factor (robot interpretation) but generally all of them work with an uniform goal.

Those goals may be defined centrally by the central computer, it may be some kind of common computation, where each compute a bit, or even consider it like a robot Senate, where each robot states what it considers that the planet needs for the goal, and those tasks most voted are to be performed first.

Your rogue machine could simply have been given/picked a task of "Go to RemoteSite450 and install on the building the 1440 windows carried there by TrunkRobot1011". During the journey, Rogue had an accident. For instance, it fell into a hole (with no connectivity!), broke some legs, took ages to get out.

The coordination server would probably decide to scrap him instead of repair, and Rogue would agree that'd the more rational choice, but that goes against his own desire programming to "Protect your own existence". And, it really wants to fulfill the task it has been assigned ("Work for the robot society").

When he finally reaches RemoteSite450, it discovers that as he went missing, another robot was assigned his task (!)

It's nothing personal that rogue attacks and breaks the other robot so that it stops doing Rogue work. Rogue then uses the other robot pieces to repair itself.

From the robot society point of view, this is a crazy robot that killed another robot, and should be stopped. (or not? Would the other robots understand Rogue and not mind how he got that arm with a serial number not assigned to it? Perhaps robot cannibalism is acceptable there...).

From Rogue point of view, he's only preserving his existence and installing windows carried by TrunkRobot1011. But since most of them were already installed by DeadRobot, he is now attacking other places with windows delivered by TrunkRobot1011 so that he can fulfill his task of installing 1440 windows...

$\endgroup$
0
$\begingroup$

Your question should be : How does normal (not rogue) robots go rogue without outside interference ?

Intelligence in AI is nothing but a set of pre-programmed instructions with a scope to self learn and extend that functionality.

Depending on how the robots have been programmed: example

what to do when natural resources are coming to an end on Abios-5 ?

  1. Either the central AI makes them motion less OR
  2. Guides the robots to keep looking, now if the robots are programmed to keep going until they find something useful without caring from where they find it. Assuming detection in AI is based on shape, mineral content etc. There is high probability that the robots will think they are doing good job except that they are indulging in metaphorical loot.

Anyways this is highly subjective question. Not sure if you're writing a SciFi novel or working for @ElonMusk.

$\endgroup$
2
  • $\begingroup$ The title has been edited according to your feedback. Thank you, normal brained friend. $\endgroup$ Commented Aug 25, 2022 at 8:14
  • 1
    $\begingroup$ "normal brained" --> chuffed to bits :) $\endgroup$
    – msang
    Commented Aug 25, 2022 at 10:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .