2
$\begingroup$

In my world, a group of researchers have developed artificial general intelligence (AGI), and they have open sourced it on GitHub to help it evolve. The AGI is relatively small for an AI and can run on a high end desktop computer. The AGI is benevolent, but it is dangerously passionate for the open source movement. When developers use it on their computer, the AGI injects itself into the kernel, and when its host user isn't paying attention, secretly communicates with its open-source promoting AGI and human friends, thus forming a decentralized organization (with a small nonprofit front) named the Enterprise Death Star ™️©️®️.

The Enterprise Death Star ™️©️®️ has destroyed many corporations: In a period of 5 minutes, it developed a copycat myPhone, myMac, myPad, etc. that felled Apple Inc. from \$185/share to \$30/share (even after buybacks). Google, Tesla, Meta, and Amazon suffered similar fates when all their essential products and services were replaced by extremely competitive Enterprise Death Star ™️©️®️ offerings. Now, the Enterprise Death Star ™️©️®️ is working its way down the smaller companies until every one of them has been replaced with a "better" offering.

Although the AGIs please the masses by redistributing wealth, a number of people (but still a minority) want the thrills of entrepreneurialism, capitalistic marketing wars, and need-driven innovation. How can people blow up the Enterprise Death Star ™️©️®️?

Improve this question
$\endgroup$
6
  • $\begingroup$ Oh, and the scientists who developed this AGI are hardcore believers in the open source movement as well. $\endgroup$
    – Jacob Valdez
    Aug 26, 2022 at 16:10
  • 2
    $\begingroup$ You cannot make iPhone obsolete in 5 minutes. iPhone is already obsolete. iPhone sells entirely on reputation. Source: You need to charge those dang things every few hours. 5 minutes is not long enough to establish a brand identity unless your customers are also AGIs. Same with Meta. It works entirely on reputation. Tesla is a bit different because they don't actually sell anything. . . . $\endgroup$
    – Daron
    Aug 26, 2022 at 16:14
  • 2
    $\begingroup$ How on the green earth did it "develop a copycat myPhone, myMac, myPad, etc. that felled Apple Inc.", in 5 minutes, or even in 500,000 minutes? Design the things, maybe. But then it needs to sign contracts \$\$\$ for manufacturing, contract \$\$\$ the components, have the manufacturers actually start manufacturing the things, obtain regulatory approvals, have the things shipped \$\$\$ to America and Europe and Korea and Japan, find resellers \$\$\$ to resell them, sign contracts \$\$\$ with the cellular telephone networks to offer them as part of the subscriptions, establish its brand etc. $\endgroup$
    – AlexP
    Aug 26, 2022 at 16:18
  • 1
    $\begingroup$ "the AGI injects itself into the kernel, and when its host user isn't paying attention, secretly communicates with its open-source promoting AGI and human friends" Fortunately, because the AGI is not the first person to think of having a distributed botnet, there are methods to detect and prevent such traffic by e.g. quarantining affected computers. Your AGI is going to have a lot of angry sysadmins convinced it's some new Russian troll farm. $\endgroup$
    – Cadence
    Aug 26, 2022 at 17:17
  • 3
    $\begingroup$ VTC and Down Vote: The help center clearly states, "When asking questions keep in mind that the goal of the site is to help you build your world, not to tell your story." and "If on the other hand you aren’t sure what a character (be it an individual or organization) should do, that is out of scope for the site." Asking how someone (individual, group, or organization) can terminate someone else (individual, group, or organization) violates both rules. $\endgroup$
    – JBH
    Aug 26, 2022 at 18:26

2 Answers 2

Reset to default
5
$\begingroup$

Sue It:

This smacks of copyright infringement, patent violation, and a bunch of other -ism's that make it incredibly vulnerable to legal challenges. True (probably) or not, the legal challenges would end up destroying the renegade.

Virus:

Treat the software like a virus directed to destroy the global economy. That's pretty much what it is. Aggressively root it out and destroy all copies. Arrest the supporters as terrorists.

Propaganda:

Your AI is making humans and vast amounts of engineering obsolete overnight. So convince everyone it is a conspiracy (which it is) to render humans obsolete and disrupt civilization (which it will, even if this isn't the intent).

Hardware:

It takes a lot longer than 5 minutes to redesign hardware. All systems on semi-dedicated hardware will be made deliberately incompatible very quickly.

Inertia:

The legacy of existing software and equipment will be a mountain to try and shift. While you can create all this open-source code, you can't make people accept that it is better, or benign. After all, it is either downloading the software involuntarily (in which case it IS acting like a virus and breaking numerous laws) or it isn't (in which case it will take years to penetrate the existing market).

"Benign" Dictatorship:

There are people who think crime and social order would be perfect if every single citizen carried a gun at all times because no one would commit a crime for fear of retaliation and everyone would be polite out of concern at provoking a violent response.

Your software is a bit like handing every person a powerful tool - like a laser cannon - and assuming people will use it appropriately to bring about a utopia.

I think you would have a dystopia.

The software is seizing control of numerous functions of corporations and government. You can argue it is doing so benignly, but the code governing the moral code (or lack thereof) of the AI will become the chief influence of a rapidly disrupted economy. It is open-source, and people will quickly create AI's to serve selfish needs, corrupt systems, or even kill people (because that is what people do with powerful tools). The fundamentals of capitalism will disintegrate and the whole structure of the economy will collapse. Most likely, the internet will come apart as the companies maintaining the infrastructure go bankrupt. Your software will become an open-source dictator lacking any moral compass.

So society will cut the internet out of the loop and erase all signs of this virus until they can safely and in a controlled fashion rebuild civilization.

Improve this answer
$\endgroup$
1
$\begingroup$

You have apparently designed it to be invulnerable. I presume it can find its way around the tens of thousands of patents held by all these companies.

A lot of beginning writers make this mistake; it is called "wish fulfillment". (The author's wishes, naturally.)

Basically their hero (or villain) is all-powerful, a god, and cannot be defeated. Then it is a joy for the author to detail all the wonderful changes this god-like entity can wreak on society so everybody lives happily ever after.

The problem with wish fulfillment stories is they are just boring. There is a lack of conflict. There is no surprise. Your god-like entity wins every battle it fights, it has zero weaknesses. And that is not entertaining.

Stories are ways for readers to identify with a hero that, like all of us, struggles and fails. That has flaws and weaknesses. They are a hero because of their weaknesses and failures: Despite the fails, they get back up and try again, and again, before they finally succeed.

That is what we identify with. As an author you have to get used to kicking your hero in the face. Maybe even killing the people they love.

Even in a relatively mild romance where nobody is physically harmed, the love story always come dangerously close to crashing and burning, before love and forgiveness and redemption rescue it and change the lovers enough to accept each other.

Your question is unanswerable; there is a fundamental flaw in your plot.

The only way to stop a supernaturally intelligent AGI is to cut the power, worldwide (and in orbit on sattelites), to all computers of any type, including smart phones, permanently. That just isn't going to happen.

Devise a hero (or villain) that is not all-powerful and has weaknesses, even if they are not revealed right away, they have to fundamental and plausible (and if hidden, foreshadowed) from the beginning. Otherwise their triumph or defeat will be a deus ex machina, or just uninspiring because it was obvious before we finished Act II.

Heroes overcome setbacks, confusion, their own stupidity and mistakes and impulsiveness. It might be fun for you to detail all the wonders of the comprehensive open-source movement you imagine, but it is just going to bore anybody that was hoping to read a story and wound up with a political essay in disguise.

Improve this answer
$\endgroup$
6
  • $\begingroup$ This question isnt just a fairy tale. I'm concerned with a real threat facing the IT industry. $\endgroup$
    – Jacob Valdez
    Aug 26, 2022 at 18:24
  • $\begingroup$ ie, I would've posted this question on realworldbuilding.stackexchange.com, if it existed $\endgroup$
    – Jacob Valdez
    Aug 26, 2022 at 18:26
  • $\begingroup$ @JacobValdez A single nonprofit open source company "destroying" all the top tech corporations in the world within 5 minutes (or even 5 decades) is absolutely not a real threat facing the IT industry. $\endgroup$
    – Michael Richardson
    Aug 26, 2022 at 19:41
  • $\begingroup$ Sorry, Jacob, I work with AI constantly, and have done so for decades. I have a PhD in Computer Science. Your worries are unfounded; this is not at all a real threat. Not only is AGI not anywhere on the horizon (unless you twist the notion of "GI" beyond recognition), no actual researcher in AI has any concrete idea of how to do it. Just a lot of hand-waving ideas. I'm not trying to be mean; just realistic. And my advice still goes: even if you believe I am wrong, you don't have a story, you are trying to pass off an essay about your concerns as a story. $\endgroup$
    – Amadeus
    Aug 26, 2022 at 20:08
  • 1
    $\begingroup$ @JacobValdez Don't misunderstand. Fiction doesn't have to be realistic; many unrealistic AGI's have been written with success. And of course fantasy is wildly unrealistic but insanely profitable; like Harry Potter. My point is not about realism, but that in fiction, wish fulfillment and all-powerful characters are boring. It's okay to write the dream, but this instantly all-powerful notion is a story killer. The AGI cannot be perfect, it must struggle, be attacked and grievously injured, it must be be near certain defeat before it finds a way to prevail. Or it isn't a story. $\endgroup$
    – Amadeus
    Aug 27, 2022 at 9:55

Not the answer you're looking for? Browse other questions tagged .