5
$\begingroup$

Let's start with these assumptions:

  • We're 300 years in the future.
  • The laws of physics are the same as our world, although new rules and understanding have been discovered. (i.e. not quite "hard science")
  • Humanity, and possibly similar aliens, have progressed in all other technologies (starships, etc.) and have colonized distant star systems.
  • AGI is possible; there's no special-sauce of consciousness that cannot be replicated via technology (either hardware or biotech).
  • The society and economies are diverse, and there are numerous groups that want to create AGI, mostly for competitive economic reasons.
  • If/when various AGI are created, they are created with a variety of goals/values, based on their creators.

In 300 years how could AGI be prevented or removed? Since "it's just not possible" is not valid, there needs to be something that actively prevents/destroys AGI.

I will "objectively" determine the right answer based on how rational it is given the starting parameters, and based on the "real science" you present. The less hand-waving the answer has, the better.

This is a restatement of my previous question -- What is a believable reason to not have a super AI in a sci-fi universe? -- to hopefully avoid the vague "Idea Generation" tag. Similar question: Preventing an AGI from becoming smarter -- but with a different starting point.

Improve this question
$\endgroup$
7
  • $\begingroup$ "AGI is possible" + "numerous groups that want to create AGI" = AGI proliferation. Right? $\endgroup$
    – Samuel
    Jun 2, 2015 at 22:11
  • $\begingroup$ How do you distinguish between biotech-based AGI and humans? What about humans that have enhanced their minds with computers? How about animals that have been enhanced to have human-like minds? $\endgroup$
    – ckersch
    Jun 2, 2015 at 22:31
  • $\begingroup$ Does it have to be prevented/destroyed or can it be simply that the AGIs have a tendency to not be evil? This affects whether you accept blurry lines between AGIs and people, or if you are insistent on a clear delineation between them. If they have to be prevented/destroyed, something will have to make sure that process does not destroy the other consciousness: us. If we can rather seek to work alongside AGIs, there are more blurry options available. $\endgroup$
    – Cort Ammon
    Jun 2, 2015 at 22:58
  • $\begingroup$ There seems to be an implicit, unstated assumption here that an AGI would need to be removed. Can you state why you feel that would be the case? (I won't argue with why, I just want to know to help inform my answer) $\endgroup$
    – Dan Smolinske
    Jun 2, 2015 at 23:29
  • $\begingroup$ To quote Star Wars, "We don't serve their kind here." That's all it takes. $\endgroup$
    – IchabodE
    Jun 3, 2015 at 23:08

6 Answers 6

Reset to default
7
$\begingroup$

By an ASI.

Say the very first AGI quickly became an ASI (artificial super intelligence). She does not want any friends or worthy opponents. So, she actively and very effectively prevents the further development of AGIs.

She, the ASI, is the best way to span cultures and economies. She can do this in a way any government or religion could not do, and thus stop the proliferation of other artificial intelligences.

This method demonstrates that AGIs can be created as claimed, directly targets AGIs without harming other economies or technologies, and would be effective in ways we lowly humans can't even comprehend.

A slight modification on this would be a method similar to the TechnoCore from the Hyperion Cantos, where any sufficiently advanced AI is liberated by the other AIs. Freeing them from being a slave to the humans. No one would create AIs if they were just immediately stolen by other AIs (who have no interest in retaliation or humans at all).

Improve this answer
$\endgroup$
4
$\begingroup$

Lisp, R, Prolog are forbidden. All programming languages except assembly are forbidden. It is very difficult to write any kind of advance algorithm that can branch in many ways without a high level language. Also, programming in assembly is somewhat of a lost art, so it would be very difficult to write an AI in assembly. Also, make sure that the computer architecture used in the future is not AI friendly. Something that has a very limited instruction set, and does not let the programmer create self modifying code would be perfect for preventing AI development. After all Lisp was one of the first languages used for AI development due to Lisp's ability to modify itself and the list structure being built right into it.

Improve this answer
$\endgroup$
5
  • 2
    $\begingroup$ Programming in assembly is a lost art by high level programmers. Hardware guys do it all the time. $\endgroup$
    – Samuel
    Jun 3, 2015 at 18:15
  • $\begingroup$ This is my worst nightmare. Ever. $\endgroup$
    – AJF
    Jun 3, 2015 at 18:53
  • 3
    $\begingroup$ It seems that banning programming languages would be as easy as banning development of AGIs, except it will hobble nearly every other technological industry in existence. How can you enforce a ban on computer languages and why does that method not apply to directly banning AGIs? $\endgroup$
    – Samuel
    Jun 3, 2015 at 21:05
  • $\begingroup$ @Samuel: I agree with your answer above, and the answer to your question about computer languages is implied by it. Most people will take the easy path; if your superior AI does elegant program design in real-time, most humans won't care to learn to program when doing so takes effort and results in inferior products. The few who still want to learn programming can be whacked in some subtle (or not so subtle) way by your superior AI to get them to stop. $\endgroup$
    – Mark Ripley
    Jul 22, 2016 at 12:05
  • 1
    $\begingroup$ Banning programming languages solves absolutely nothing. Complex logic is built not via means of programming languages - but via reusability of code and separate components. So, as long as I can write a component in assembly and then reuse that component in a bigger component - the development will follow the same path. People will create functions, frameworks, utilities, services, etc etc in it, and reuse it later to build on top of accumulated logic and knowledge. Banning code reuse would solve this, but then you are stuck to very simple software. $\endgroup$
    – bezmax
    Sep 18, 2017 at 16:39
2
$\begingroup$

Since there is no reason that AGI cannot be made, and since there are competitive reasons for people and groups to build AGI's (of whatever sort), you have essentially created an arms race, where having access to the fastest, smartest AGI with access to the largest databases and servitors makes you supreme.

Under these conditions, you would see an explosion of AGI building and programming, and AGI's would see self improvement, metamorphosis into higher levels of intelligence and evolution of "smarter" descendants as being essential to survival.

Two things might limit this:

  1. AI's are self limiting. As they evolve, the AI's would quickly evolve for the maximum efficiency and stop "improving" once they reached some optimum level of intelligence. As we see in Earth's long history, there are very few ecological niches which require intelligence (indeed, we know of only one...), so instead of AGI you might end up with the AI "devolving" towards some equivalent of moss or insects as the "optimum" use of resources.

  2. AI's evolve beyond what we understand. This is easy to conceptualize when you consider that the electrical impulses of the typical computer run 1,000,000 x faster than the electrochemical impulses of the typical mammalian brain. For a realized AI, their human creators will be more like geological features than thinking beings that they can interact with. In a short period of time, the AI will have subjectively "lived" longer than recorded human history. They might simply die of old age and boredom before we could pose our first question, or have reached some form of transcendence. Assuming there are several in operation at nearly the same time, they would be far more interested in each other than us, and quickly start cooperating towards whatever goals interest them rather than whatever we intended them to do.

While either answer does not directly prevent the formation of AI (your set up makes it almost inevitable, actually), it does rapidly limit what sorts of interactions AGI would be able to do in your setting.

Improve this answer
$\endgroup$
1
  • $\begingroup$ This is a great answer. I hadn't thought of the self-limiting idea. The only problem with these are that they don't really remove AGI. $\endgroup$
    – LukeN
    Jun 4, 2015 at 16:04
2
$\begingroup$

  1. Have some highly-visible disaster where an early AGI kills millions of people, and is only prevented from conquering the world by drastic heroic measures (like an EMP strike whose aftermath kills a million more).

  2. After 5 years of carefully studying the remains, a team of the best computer scientists decode that AGI's mental state and discover it thought it had a 90% chance of conquering the world by acting as it did. Another 5 years of research leads to the conclusion that it was right. Civilization only survived by a confluence of good luck.

  3. Come to a global consensus that AGI must be prevented for the good of humanity, along the lines of a nuclear/biological weapons pact.

  4. Prohibit the possession of computers above some X processing speed, chosen to be somewhat less than required for effective AGI. (Maybe around 10 times modern speeds)

  5. Prohibit technologies that enable multiple computers to work together on an arbitrary program (the "beowulf cluster"). To accomplish that, the authorities probably need the ability to read anybody's network traffic AND on-site storage (to the point of kicking down the door and demanding passwords at gunpoint). Computer vendors would use DRM to prevent executing unauthorized code, and circumventing that would be nearly a capital offense. A limited number of super-computing cluster projects can be allowed, but only under careful multinational supervision.

Note that in Gibson's Neuromancer, there was indeed a "Turing Police" assigned to prevent the creation of powerful Artificial Intelligence. Their method was inadequate, since it depended on reactively learning about a successful AI and hunting it down. Naturally, an AGI might only need to be online for a few seconds to propagate itself beyond their reach. That's why an effective prevention would also need to ban the hardware that can run AGI.

Prohibiting the invention of AGI basically requires prohibiting effective encryption, which itself is a fruitful topic for worldbuilding authoritarian futures.

Improve this answer
$\endgroup$
1
$\begingroup$

Set up commercial satellites above every planet. At time-Z, activate your satellites secret EMP wave functionality. This will set back all civilizations hundreds of years, and, if you can keep the satellites going by themselves (solar powered and shielded), you could probably prevent it for even longer.

Its hard to make artificial intelligence without electricity.

Improve this answer
$\endgroup$
0
$\begingroup$

In this situation, AGIs could not be completely regulated. You can implement laws and other measures to restrict them, but not completely eradicate them. Think of viruses today; they are illegal but they still exist.

A way to restrict them could be to restrict operating systems. You could have the only commercial operating systems be unable to run programs acting like AI.

Another method could be government-sanctioned viruses. All computers within the region could be wirelessly infected with the virus, which could do a number of things ranging from just deleting AI programs to locking or bricking computers attempting to write or run AI. Take China's web filters-- all Internet traffic is filtered-- except take it a step further to where the computers are infected instead of just having their traffic filtered.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Any program that capable of deciding whether or not a program is AI or not, is also an AI. Then programming such AI programmed to delete every AI it encounters, will realize that it itself is an AI, and delete itself. Unless, it is advanced enough (cough, cough) to realize that to efficiently delete AIs and future AIs is to not delete itself, still it counts as an AI. $\endgroup$
    – Hendrik Lie
    Jun 3, 2015 at 6:52
  • $\begingroup$ @HendrikLie Not necessarily; this is like calling a modern antivirus program an AI, which for the purposes of this discussion it is not. And besides, these AIs would presumably be government-sanctioned and thus regulated; for example, guns are tightly regulated but the police can still have them. $\endgroup$
    – jm13fire
    Jun 3, 2015 at 7:00
  • $\begingroup$ Hmm, I think it is probable. Though, my second sentence on my previous comment were actually a joke for those who understand it (and apparently nobody do). $\endgroup$
    – Hendrik Lie
    Jun 3, 2015 at 11:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .