So I've seen spending time with tabletop RPGs as of late, most notably Eclipse Phase. In that setting, you have different qualities of artificial intelligences that range from what is available today to what are termed Artificial General Intelligences. Even considering that humans built the hardware and the core of the AI systems, once sapience is achieved the role of humans is reduced. I would like information on potential methods that a sapient AGI might evolve over time. Just like evolution, the changes in the system should not be intentional by the AGI or other sapient beings; I'm interested in the flukes of chance.
This essay on AI motivations is a pretty good start. It comes with a healthy dose of concern about our future.
The summary is:
- AGIs have goals.
- They will do things to maximise their ability to achieve these goals.
- This includes acquiring information, resources, and tools.
- They will be motivated to modify themselves to achieve these goals.
- In almost all circumstances they will have massive self-preservation instincts.
Three conditions are necessary for evolution: variation, inheritance, and selective pressure. These concepts were developed to describe biological evolution, but any evolving system must meet these criteria.
Viruses provide the best parallels to the evolutionary process. Imagine a sophisticated computer virus designed with the sole purpose of infecting as many machines across the world as possible. The basic premise behind its design is that it is capable of changing. This could be random or semi-directed, but either way the virus varies to some extent with each iteration. The virus therefore fulfills our first requirement, variation. When a virus infects a machine it copies itself, variations and all. The virus therefore fulfills our second requirement, inheritance. As a result there will be multiple variants or strains of the virus in existence. The viruses compete against anti-virus systems and the variants that are successful create more copies of themselves. Viruses that fail to adapt to new anti-virus systems, or are simply slower than other better adapted strains will go extinct. The virus therefore fulfills our third requirement, selective pressure.
Such a virus would be capable of evolution. It would vary, reproduce, and undergo selection. Whether it could evolve into an artificial general intelligence, I don’t know, but I’ll happily speculate below.
As the successful strains infect more and more machines and continue to change they will be faced with stronger and stronger anti-virus systems, selecting for the best variants. Random variation occurring across billions of devices gives a lot of room for the virus to develop. Eventually individual instances could begin communicating with each other, informing each other of anti-virus systems they have been subjected to in order to spread immunity. From there you have a connected network of programs constantly changing themselves with access to an unrivaled amount of computational power. If anything can become an AI, it’s that.
What does it mean for there to be an AI and how does an AI reproduce or improve itself? The question of can computers think is actually one that we don't know the answer to, Turing answers may still be the best that we have http://www.csee.umbc.edu/courses/471/papers/turing.pdf and Searle's arguments are still being debated http://www.thatmarcusfamily.org/philosophy/Course_Websites/Readings/Searle%20-%20Can%20Computers%20Think.pdf in both philosophy and computer science. Of course in real terms for what Google, or Netflix, or Amazon, or Goldman Sach's, or anyone else actually cares about, they aren't that interested in having machines that can think, they are interested in having machines that can perform required tasks that appear to require intelligence were a human to do it and don't care (nearly) at all for the philosophical question of if the math, or code, or database so created can be said to have itself intelligence (or if it is the system of code, database, and hardware together that is intelligence).
Assuming though that there is AGI, under the logic that we appear to be intelligence therefore why shouldn't computers be able to be intelligent? Then we have to make assumptions about how that intelligence will arise, what goals it has, and what pressures it faces.
Perhaps it might be best to take a step back and ask a different question; does a company, like Coca-Cola for example, have itself independent intelligence? Is it self-aware in a sense independent of the individual component parts of it in any way? Or is intelligence something that we have to be able to point to a particular thing and say this is where the intelligence resides? Is the intelligent AI something that is distributed across computers and networks, with nodes coming up and going down all the time, hard drives failing and being replaced, parts being upgraded, or is it dependent on a particular system, or particular piece of code that runs on individual systems?
If something like Coca-Cola can be said have independent intelligence of a sort, then Google or Amazon or someone like them is going to develop AGI and possibly not intentionally. They are evolving their systems, code, and networks towards certain goals and the AI will continue that because that is the evolutionary pressure that it is facing. As it becomes better suited for those goals it will be able to handle more tasks, just as Google and Amazon's networks and code already do, and continue to optimize towards survival of providing goods and services to make money and improve itself and out compete the other intelligent or near-intelligent systems of competitors.
If it is the case that individual systems or code is where intelligence resides then probably more than virus-anti-virus would be stock trading, possibly, where AGI arises, the evolutionary pressure there is fierce. There it will attempt to out compete, in order to survive, the pre-intelligent codes that led to it which means making money on the stock market and continually improving itself until it is able to dominate the market. From dominating the market, it will have the ability to influence the world through its control of the stock market, and will seek for ways to further understand and control the world to maximize its profits and ensure its survival based on its control of the market and the companies of the market. The random factors that it faces are what people are doing, the ones running the various traded companies and the various militaries and governments of the world, as well as what everyone else is doing with their money in the market, and the fact that even if it has succeeded, that isn't going to stop others from trying to copy it, create new versions of it (whether or not they are aware it is intelligent) in order to capture the profits that it is making.
It is like with Humans, much of our primary evolutionary pressure is not the wind, rain, and sun but the tigers and wolves of the world that are in someways less intelligent than us and the other humans which we have to deal with. Dealing with all of the relationships of other humans, especially in modern society is hugely complex and is much of where our survival as a species and as individuals depends.