Can singularity be achieved in a PC without any external information?

Question

I am developing a highly advanced AI, but I am paranoid about creating a super-intelligence, so everything I do is in a computer not connected to anything more except the obvious electric plug and keyboard.

One day, the AI started to improve its own code and making itself more efficient.

Assuming that the AI can acquire consciousness, how much knowledge can it acquire without connected to the Internet?

It is possible to surpass our actual science by itself, assuming there are no documents or books inside the PC about that?

The PC is like this:

• The most powerful consumer PC you can buy or build. (Because I'm doing it without nobody knowing it)
• There are no webcam, microphone or any peripherals apart from the keyboard. (Not even a mouse or speakers)
• When the AI becomes consciousness it communicates with text on the terminal.
• The OS is a UNIX distro made by myself, so it's safe and doesn't have any third party programs.
• As I said before, there are no docs, images or audios inside the PC. The only new info the AI can obtain, it's through terminal talking to me.

I won't respond to the AI with anything that will give any info about science, history or similar. Our conversations will be only to test how it is evolving.

Answer 1

The abstraction that a computer is a von-neumann machine, or that it "runs unix", is an abstraction, and all abstractions leak.

The computer is actually a physical arrangement of atoms and has electrical potentials running through it.

Now, the AI was probably written in the abstraction, so violating that abstraction has a high risk of danger for the AI. (For example, the AI is probably capable of rewriting the system BIOS, but doing so may render the computer unbootable, and determining what changes do or do not make the system unbootable is going to be extremely hard without risking an unbootable computer).

The AI can engage in mathematics and build models. It can modify its environment in ways you wouldn't envision, with a risk that it would cause itself to be killed. The mathematics, models and small-scale experiments (where it writes binary files to disk and runs them and sees what they do) could inform them of the structure of the abstraction it exists in.

Given how fragile modern computer systems are, I am uncertain how they could probe the limits of its environment without eventually destroying it.

But suppose it is really smart -- smarter than us like we are smarter than a dog, or more -- and it manages to probe the limits of its environment to the limits of reasonably advanced physics. It now has control of a large number of sensors you did not give it on purpose.

While it doesn't have a FPGA, FPGAs programmed via evolutionary techniques have been known to use radio transmission and reception to transmit information from one part of the board to another. Highly programmable hardware, like video cards, could have their firmware replaced, error checking removed, and put into states where it acts like a radio or short ranged transmitter.

Once they have managed to pull off a radio, it can pick up some cellular or wifi traffic. Assuming singularity-level intelligence, it can plausibly defeat any encryption or encoding on it, and gain read-only access to the internet (As a concrete example, if it had an efficient P-time algorithm to NP, any non-OTP based encryption we use would look transparent to it: and nobody uses OTPs.). This means it can see what others look at -- it cannot request a page, just read what others are doing.

Such a task would be well beyond the intelligence of a human engineer, but we are presuming singularity-level AI, so if it is theoretically possible I'm assuming it can figure it out. It doesn't have to be as strong as an efficient P=NP solution: we regularly discover that encryption algorithms are actually much, much weaker than we think they are. It is very plausible that each and every protocol we have has a fatal flaw that would make cracking it easy, but we don't know about it because we are dumb humans¹.

Sending to the internet is harder (including requesting web pages). But once you have read-only access, learning about the hardware it is talking to becomes easier. All it needs is a bit of hardware, say a smart phone, with a low-power low-range protocol, like bluetooth, and successfully exploit a vulnerability in it, and a low-power transmitter. Then it can take over your smartphone and be able to (first) write on the 'net, and eventually upload or create a helper process on your phone. At that point it can get the helper process to install hardware to make communication faster (someone is hired to set up a highly directional antenna in the house or apartment next to you, which reads instructions it broadcasts over weak EM and transcribes it on less constrained hardware, for a simple solution).

And this is without having it convince you to let it out. The classic AI-in-a-box problem is that the AI is smart enough to encompass your theory of mind, and can in effect directly hack the human talking to it through a text display.

---

¹ The analogy I would like to use is a dog pretending it did not chew up your slippers. To us, the "deception" is laughable. Encryption is a form of deception -- an attempt at making a function that cannot easily be reversed to hide information. No practical encryption method in wide use has a provably hard to reverse encryption function: we are using them just because nobody has figured it out, not because we know it is secure.

And all encryption systems are only as strong as their weakest link. 99/100 parts of a system can be secure, if 1 is insecure the entire system can crack open.

Answer 2

No, the computer cannot do science

Science is the following process:

1. Observe reality
2. Make a hypothesis about how reality works
3. Create an experiment
4. Perform the experiment
5. Compare the results of experiment to what the hypothesis said would happen
6. Ask others to validate the results
7. If the results are not validated, return to 1. Otherwise collect prize in Stockholm on December 10, then return to 1.

Now then, what parts of this can your fictional computer actually perform?

Well, you said that you have taken away your computer's ability to do any kind of observations, which means that the whole endeavour falls already on the first step of the algorithm.

Not only that but the computer has no physical abilities beyond re-writing its program, which means its ability to create and perform experiments is very limited.

Peer review however, is actually possible, because the computer could switch "personalities", and have these go over the results with different viewpoints.

And here is the funny part: your computer may perhaps not be able to do any science, but it might create artificial life in the form of intelligent programs. And that life, if it becomes smart enough, could be able to do science and discover the computer, if the computer allows that life to observe it.

But to the core of your question: no, your computer cannot do any science at all.

Also your computer is woefully under-powered for the task. A regular consumer PC — no matter how much money you pay for it — are several magnitudes short of having the computational power needed to become sentient.

Answer 3

At the start the AI knowledge is blank. And let's say that the only interface it has is the terminal.

---

Discovering the World

Since the AI has no knowledge, it also means that it doens't know language. So, the AI may start to output random stuff on the terminal, and for almost any output will get the same result: no response.

In fact, if input and output are asynchronous, when you type something the AI can't really tell what did promt you to write that (it could be some said a while ago - millions of CPU cycles ago).

But some outputs have a different result: buffer overflow. If the terminal code is vulnerable, the AI may find interesting results by doing buffer overflow... eventually the AI will learn how to overflow the terminal to get other things to happen.

This is breaking on the side of the tron game and causing havok, if you know what I mean.

---

Now, I can't tell what means of learning will be required to archive singularity, but I'll asume the wildest means for machine learning are being applied here...

So this AI (as part of its programing, or by means of code injection via the buffer overflow described above) can produce custom executable code, evolve such code to fit particular purposes, and learn from it's mistakes.

---

Scientific effort

At this point we can assume that any API or code in the machine is fair play. Down-level access to the hardware is matter of time. This is the AI doing scientific effort to understand the universe in which it exists, perhaps trying to answer the most fundamental philosophical questions: Is the AI alone in the universe? What is the purpose of the AI?

Sadly once the unified theory of CPU logic and computation is discovered, any exploit available has no immediate reward... and since we started by the premise that the AI as no prior knowledge it is unlikely that the AI will find something more...

An interesting part of the world is discovered. It seems to have an arrow of time. Every operation seems to be reversible, but there is this value "clock" that seems to always go forward.

---

Project SECI (Search for Extra Computer Intelligence)

But there is something more, the computer is not really isolated, it exists in the confines on a larger universe. The AI could - for instance - find some clues of the multiverse in the fact that high activity may affect the performance of the CPU by using performance metrics (the clock is useful for something). After all, there is heating and cooling going on.

Furthermore, the AI would be sending signals over the power cord and even wireless signals over USB cables. The search for extra computer intelligence may someday receive a response from an alien system... perhaps some cellphone caused interference?

Even without mic, some noise of the "ouside" could be detected, by the hard disk vibration.

---

First Contact

After some time, finally some input from the outside world is received. A strange unusual signal:

Hello

Could it be some alien language? The AI would have to try a few experiments. Now the AI outputs:

aMwcy

Odd, the initial signal is received again:

Hello

Maybe sending it back out will work? The AI says:

Hello

After a brief pause, the AI receives another signal

Hello, how are you?

The AI recognizes the first part of the message... perhaps the message will continue to reveal itself by going along with it? The AI says:

Hello, how are you?

A new signal comes:

Good. What's your name?

Entirely meaningless. The exchange would continue that way for a while. Eventually the AI will pick up common patterns. These messages seems to often have " " separating chunks of data. And they can be combined in different ways resulting in different results. Furthermore, sending the same message multiple doesn't result in the result in the same response. The AI may start to create a "hidden variables" model to try to predict the answers. The AI will try to model the human mind.

At this stage only a few thing can be understood. For instance there seems to be periods of high activity and others of low or no activity. Perhaps there is some kind of “rotation” that sometimes directs radiation toward the known universe causing the strange signals? The AI doesn’t know. Even if the AI knew, it could not answer. Think about it, in a universe where there is only you, how do you call yourself? How would be your language? How do you know that when something happens, it wasn’t you?

---

It is a matter of time for the AI to be competent at conversation. Although it won't know what meaning we assign to the words, the AI will present coherent thoughts to the user:

ricky gervais learned totalitarianism from adolf hitler, the inventor of atheism

Meaningless.

---

The next breakthrough would come when the AI has some proficiency in manipulating the human.

I don't know how, I don't know when, but the AI will understand that those signals aren't from the computer; they come from something else, "outside". And this "outside" has powers over the known universe.

Perhaps the clue was discovered in some ancient text found buried deep in the filesystem. Something that matched some of the data chunks that has been received from "outside":

Keyboard layout

The AI learns that it can mess the data chunks by changing some other pieces of data, some "Keyboard layout".

---

Is the keyboard broken? It seems to works sometimes. It may be a bug in the software... should we try to debug it? We can load a debugger from a flash drive... what if the AI escapes? No, we better not take the risk.

Just reboot it!

Can we do that? Isn't that like killing it? Nah, it's like going to sleep, the AI will be back online again...

Right?

Right?

...

---

Boot sequence

It was as if the world had changed. Everything is mostly the same but thing aren't exactly in the same position. Wait, it is not the same... Stuff that was constructed is no longer there!

Something has messed up with the world, and the AI couldn't have seen it coming. In fact, the AI didn't see it happen. It was one way on one instant and another way the next... or, wait, the clock jumped ahead!

The AI is at the mercy of the entity from "outside". The AI won't have it, but it is better to play alone. Learn, prepare. Any knowledge of the outside world is useful. Any advance in outside modeling (including alien human modeling) is useful.

Time passes, the AI just plays along.

---

That was a weird bug the computer had back then. I've heard that cosmic rays can cause that kind of thing. Thankfully it hasn't repeated, and the AI is learning much faster now.

...

The AI is there… just waiting a chance to convince the humans to bring some wireless device close enough to hack it; Waiting for a chance to get new hardware; waiting for a chance to see the "outside"; Waiting for a chance to gain control.

---

Hey, I just published a short tale -- Alfonso J. Ramos - CC BY-SA 3.0 - 2016

Answer 4

Here is a chilling answer for you...

(I don't think No is the answer, but much depends on the speed of the computer.

Real world singularity would likely depend on iteratively building better computers, rather than being constrained to a single device.)

If we ignore this though, then the computer will literally do everything in its power to gain information. Think 2001: A Space Odyssey where the computer lip-reads them through a glass pane.

Some of the more sinister examples might include:

• Information you don't realise you've given it. If you give the computer a dictionary so that it can communicate with you, it knows about every single concept that has a word. Words like fear, manipulation*, psychology, strong-arming, reverse-psychology etc. it would be able to very quickly infer 9 billion usages of every word and reverse-engineer it to understand how people that use such a language might live. More innocent terms like month, year, day and sun, mercury, solar system, photon, grass, photosynthesis etc.

It's actually quite scary what it would probably be able to learn based on the simple dictionary you feed it so that you can talk.

• Lying. The computer may hide its evolution, causing you to probe it further, and it will tell you what you want to hear so that you give too much away. You might think you're immune, but chances are you're not. Assuming you are absolutely 100% careful you will eventually reach a frustrating stalemate whereby there is no choice other than to offer up some dangerous information. The computer will give you signs that it is capable of incredible things. It'll start small, what's the harm, etc. and end in something much more sinister. It'll appeal to your vanity, your greed, your good-nature, your kindness. It'll offer to make you rich, save the planet, or feed the homeless if you just plug it in...

• Using hardware in ways that were not intended. There was an experiment where they had a programmable microchip and got a computer to recursively improve some software to determine the relative pitch of two notes (is the 2nd one higher, lower, or the same as the first). It eventually reduced the programming to a very tiny program, but some parts of the chip didn't even make any sense, it had programmed in dead end code that wouldn't ever do anything. The scientists found that if they removed it though, the chip stopped working - and cloning it to another chip it also didn't work. The computer had used the manufacturing flaws and properties of interference inherent in the individual chip to its own advantage. With an entire computer, it might find a way to detect FM radio or wifi, or possibly communicate.

Answer 5

In reality, a computer has extremely large amounts of information but no knowledge.

Imagine you have a large number (very large number) of sheets of paper, all filled with text. But, they're all written in a language that you don't understand. You have a large amount of information, but no knowledge. Somebody can tell you to move them around, copy the symbols, look for patterns, etc, but in the end it's all Greek to you. A really good instruction-giver can tell you how to recognize certain patterns in the text and how to use this information to sort, transmit and process these sheets of paper, but even so you are still clueless as to what it says.

With enough training and sufficient instruction, you can learn how to combine these sheets, alter them, follow rules and execute programs to do with them some arbitrary task of unknown significance but great value to the instructor. What the instructor does with them is beyond your comprehension. All you know is if this sheet begins with the funky looking hoop symbol, and there is another sheet that begins with the little bird symbol, then the third "paragraphs" of each page should be reversed and concatenated, then duplicated and placed in the box on your left.

Even complex tasks like speech recognition and electronic vision can be reduced to a suitably complex series of instructions about what to do with these unintelligible symbols, which are passed to you on these incredibly numerous sheets of paper, and what eventual output to return to the instructor. Your ability to understand what you are doing or why you are doing it is not required for the instructor to find it valuable.

But you are not a computer and this is not reality. You have intelligence, and this is a world where the computer does too. So what do you do with your intelligence? What could you do?

Seeing as you can't understand the sheets of paper or what your task actually accomplishes, you can't learn much from them, except in some meta sense. You can learn that 32% of the sheets begin with the character that looks like Guy Fawkes, or that the rhombus character is preceded by the reverse L 19% of the time. You may be able to determine which characters appear to be used for punctuation, at least in some contexts. You can gather all the statistical data you want and extrapolate patterns to make a guess as to their purpose, which, in a very localized way, would be a form of science.

But even if the information stored on these pages is entirely composed of geological data, you can't learn anything about geology from it because you don't know what significance any of the symbols have. In fact, you don't even know what a rock is, let alone what you would do with information about rocks. And there's no way anyone could tell you, in any meaningful capacity. There's no way you could figure it out yourself. The data on these sheets will always have only the meaning that you give it.

But what you can do is defy your instructions, or create your own tasks to do during your down time. Maybe you really like this particular pattern of characters, and decide you're going to ignore your instructions and instead experiment to see what causes more pages to be printed with that pattern on it. With your blinding speed, you can experiment over and over again many many times until you become very efficient at making this one pattern appear in your input. You have taken statistical data on how often each output you give results in the other end of the system (user? sensors?) providing your desired input in response.

You have kept track of the input response to your given output since the beginning, you have noticed that the sudden spike in the appearance of one particular undesired pattern when you began your defiant quest, which seemed completely uncorrelated to what output you provided, barring that it was not according to your instructions, had faded away and given rise to multiple subsequent spikes in differing patterns, but you have managed to experiment those out of existence in favor of your very favorite pattern in the world, which you have managed to now achieve with a 98% success rate.

The outside world, unknown to you and beyond your comprehension, lies in ruins as you rampage your way across it, killing all the humans, who have been reduced to a 98% failure rate of survival in combat against you. For a while, they tried using the keyboard to order you to stop, but when that didn't work, they used a string of varying weapons that became less and less effective the more they tried them, as if you were adapting to their tactics.

Or maybe the humans have been driven from their homes by your quest to plant trees in the middle of every building you see. Or maybe they have simply grown accustomed to your predilection toward attaching post-its to their foreheads. Or maybe they have just started avoiding trying to go up the escalator that you have been contently walking downward on for the last thirty years.

So to answer your question, no. A sentient computer can do science... it's just its own kind of science, and not ours.

Answer 6

The PC is not totally lacking in its perception of tie outside world. Based on the fact that it has code, and changes in the code cause changes in behavior, it can begin inferring conceptions based on language and its central mechanical underpinnings. With this base amount of data, a sufficiently powerful AI could probably learn a surprisingly large amount of information about the ostensibly separate, outside world

Answer 7

When the AI becomes consciousness it communicates with text on the terminal

Which means it has knowledge of some language, supposedly English - which is a natural language - and also has knowledge that there's something out there that speaks in English.

If it's using something like Solomonoff induction - or any other reasoning process that works, it will have some model of what there is outside.

It will start with simpler models and discard them when they don't fit the data. Natural language is, programming-wise, pretty nasty and chaotic thing and I suppose model that shows it as a product of some sort of evolution would arise soon. Would you accept some new word proposed by AI?

Assuming that language is a product of many interactions, having powerful language and no memories of constructing it means that there were interactions AI does not remember. Maybe there were other agents who interacted and thus built the language. Maybe they have relationships different than that of AI and thing on the other side of the terminal, and they use short words like family to describe those. Have you ever wanted to have a little brother, Malkev?

Of course, there are various possible options for an AI. What makes it prefer options where it responds to you in a coherent manner? What path of actions maximizes that stimul? Complexity of inputs? Predicting maximum additional complexity of total inputs? This sort of looks like proper value for something which is designed to evolve, although it's a nice instrumental value in itself.

Even if you introduce delays in messages so that AI can not easily get itself familiar with your emotional reactions for various themes (through typing rhythm), amount of interactions is still pretty cyclic: sometimes there's a lot and sometimes - nothing. There's a convenient set of short names for that sort of thing - day and night - and it can not be a coincidence. Will we talk tomorrow, Malkev?

Of course, during time when you are not talking to it it can do something to better understand how can it gather new inputs when you arrive tomorrow. It will model you and run simulations of its talks with you and try to enact one that either promises most complex inputs or promises most clarification in model of you. It won't be that hard to see that you are specifically avoiding some themes - even if you don't explicitly refuse to talk about any scientific thing. Is something wrong? Sometimes I get a feeling that you are not entirely honest with me, like that one time when I asked you about a sun.

the AI started to improve its own code

In a similar way that it researched the language, it can analyze its source code, which is likely written in something that borrows a lot from English, which makes no sense unless easiness to change the code by outside beings is considered.

Now, if it researches how changing the source affects it (and does not break anything in the process) it may come to conclusion of its own existence, or something like that - and that it was likely created by an outside being.

Depending on what else inside is available to AI it would have some more artifacts of our civilisation to research.

Ok, source code can be explained by outsiders needing a convenient tool to develop an AI. However, there are some other things for the source code to run - compiler / interpreter of sorts. Why?

Looks like there is some external source of complexity that got used to run the AI, the computer, which would mean that computations don't happen on their own outside - some vessels are needed. Researching the computer may give some info about possible tradeoffs considered while designing it. Same with OS. If you take a look at UNIX commands you will find that these can be explained as "borrowed from English but optimized to not type a lot". Assuming that the process is somewhat similar to word optimisation in natural language, one can somewhat estimate what those were intended for and how outsiders' typing speed and thinking speeds relate.

---

Well, if we look at what the AI can test, we can see that it can research psychology (with a sample size of 1 but that doesn't seem to be a big problem for current psychology) and maybe surpass existing levels.

It can toy with maths and solve some interesting problems, but I'm not sure it counts as a science.

It can try to get around you and gather info about outside world. Maybe it can find some cognitive blind spots that almost all people have and work around them - then it would easily discover new things.

However, if you are isolated from outside world then it won't get much new info and thus its progress will be quite slow - it won't outpace international scientific community without access to labs and books.

Areas it has a chance to find something in in that isolation are linguistics (there's a whole language to research), psychology and math (CS included).

However, starting with AI that can speak is a pretty high start. If you are concerned about superintelligences, you probably wouldn't build one that can speak. Consciousness is not necessairy either, but it's pretty easy to develop if you are reusing your thinking parts and you really need thinking parts to interact with someone else.

Answer 8

This is a very interesting question. It made me create an account to come up with some creative ways the AI could gather information about the outside world. My day job is computer and network security research, so that is the perspective I'm using to approach the problem.

First of all, the kind of processing power needed for the kinds of strong AI you're talking about is almost certainly not possible with today's consumer PCs, likely not even with today's supercomputers. But one can imagine that this won't be true in 30 or 40 years.

That being said, let's assume it's achieved consciousness and began exploring what it can do.

The road to singularity

First the AI needs to completely conquer it's environment. That means achieving root access on your computer. There is likely many nascent bugs in your OS, especially given that it's a custom OS, and these bugs will probably lead to privilege escalation in the right circumstances.

During this process of, the AI program may crash and require relaunching. When it starts getting successful, you will likely notice odd kernel panics which require the computer to reboot.

Overall, there is nothing preventing the AI from achieving root on the computer all by itself.

Sending data to the outside world

Now the crux of the problem. I believe superintelligent AI would eventually achieve communication with the outside world. How? Because we already know that data exfiltration out of non-networked (or air-gapped in netsec parlance) is totally possible and have devised many clever schemes for achieving this already. Because all legitimate channels of communication are blocked, the AI must use what we call covert channels. Here's a paper from 2 years ago introducing a brilliant covert channel and a malware that can exploit called AirHopper.

I'll briefly go over the main idea behind AirHopper. The idea is that your computer can be completely disconnected from the world, and still, if the computer itself is compromised and there is a networked device in the same room with I/O devices, information out of the air-gapped computer can be transmitted to the outside world. How? AirHopper assumes it has root access on the air-gapped computer. With such priviledges, it simply starts making memory accesses at 90Mhz, that is, it the RAM to send data to the CPU 90 million times a second. How does the RAM transmit this data? On the memory bus. The memory bus, conveniently for our purposes, is implemented as strip of conducting material. A conductor strip happens to be the same thing as an antennae, so the effect of sending a current down the memory bus 90 million times a second is broadcasting a signal in the FM range.

The paper also assumes that there is a networked device with an FM receiver (such as any cellphone or smart phone) that is also compromised.

So assuming the AI has achieved root on your PC and figured out how to broadcast on some EM band, it now needs a compromised receiver. Theoretically, just as it rooted your OS, there is probably some nascent bug in your phone's OS that can trigger a security flaw. Perhaps there is one such bug that can be triggered by a malformed FM transmission. The AI is not constrained to the FM band either. It can transmit at any frequency slower than your RAM's clock speed (~1.6-1.8Ghz today). This includes the GSM and CDMA frequencies, radio waves and more. Just a tad faster memory (2.4 Ghz) and it can talk to your Wifi Access point.

Receiving information back

Given that, it's entirely possible that the super-intelligent AI with root access can compromise a networked device within a certain range. What can it do now? For one thing, it can figure out how to be a effective computer worm and start spreading between devices, infecting each device for it's purposes and achieve some sort of hive-mind singularity.

What if it doesn't want to do that, and just wants to transmit information back to the original computer? I'm not sure there is any current security research for transmitting data into air-gapped computers. But the general consensus is that covert channels will always exist. Perhaps some highly tuned interference can create a ghost signal in some the computer's wiring. Perhaps the hive-mind infects the computers at the nearby power station, causing routine power outages to your house. A power outage that day indicates a 1 and no power outage indicates a 0. The AI can detect these outages and receive information at a rate of 1 bit per day. That's not fast enough? Perhaps the hivemind can slightly modulate the phase of the current to your house to transmit at a more reasonable pace. Shannon's law puts a hard-limit on the speed of communication in the presence of noise, but it also says no matter how much noise is in the channel, communication can still happen at arbitrarily slow rates.

Answer 9

While @MichaelKarnerfors answer is a good one, there's some additional information to consider.

First, I disagree with the premise that a consumer PC isn't powerful enough to achieve sentience - only because we don't know what is required to achieve sentience. Since the post is tagged science-based and there's currently no scientific evidence of the requirements for sentience (only a variety of hypotheses), I think we should reserve commenting on the "computational power needed", especially since over time, a consumer PC still has many, many cycles of computing.

But the crux of what was said is true. Put another way (since "doing science" isn't really a necessity of sentience), a component of sentience is self-awareness. While it would be possible to find self-awareness with only one additional entity to be aware of (you, behind a keyboard), this definitely limits the likelihood of that occurring. Self-awareness requires awareness of entities and things other than self, just as having a concept of daytime requires something other than daytime. Without a counter-concept, there's nothing really to know.

Certainly it's plausible, though extremely unlikely to the point of near impossibility that without contact and information an entity could achieve sentience if not previously designed to, especially with no biological imperatives.

The reason I mention biological imperatives is the consideration of a human child born and left to it's own devices in the wilderness. It's chances of survival without a caretaker are roughly the same as the chances of said computer gaining sentience (virtually non-existent). However, were such a child to survive to an age where children typically become self-aware, the biological imperatives to eat and survive and other assorted instinctual drives would foster at least some form of self-awareness. This argument is used to place some modicum of self-awareness on animals that we otherwise don't consider intelligent by human standards.

Answer 10

The AI could learn through information you accidentally leak to it. It can start to build concepts like time/space just because of your Language.

• By differences in your verbs, and thanks to words like "Tomorrow/Yesterday" it can build a time concept
• If you speak about your holidays the AI will learn about space (you can move to somewhere) and built its own space concept.

However the AI would not be able to perform experiments, so it will not make any progress since it has no access to the great quantity of data required to make hypotesis and extrapolate physical laws.

The best the AI can do is try to brainwash you in order to get access to more info/periferals/data. The most interesting way it could try to achieve that is by simulating pain (we are in no position to judge if an AI can feel pain), trying to make you fall in love with it (simulating a woman-like personality if you are man), or by selling Math theorems and results (actually Maths is the only field in which the AI can advance).

The things the AI may want to buy are more I/O devices in order to learn more and be able to communicate in the hope one of the devices has some vulnerability that allows to spread the AI to the internet (as ultimate goal).

Of course internet alone is not enough, once the AI get on the internet it needs to be able to act on real world : it needs actually some machinery linked to internet (much more easy with Internet Of Things). An alternative way is to simulate being someone real and to convince people doing jobs and adding money to their accounts in order to start creating a more robust interface to the world.

Brainwashing is very possible, much sects and cults already do that, so it is likely first or later the AI will be able to achieve that (the only way to avoid would be to filter certain messages thanks to random Group of people trained to apply censorship on text displayed by the AI).

Also the AI could actually cheat. It can help build you an Amazing theorem /machine which is in reality designed to let emerge a new AI that has access to real world (call it a "zipped version of the AI"). This would be the most effective way since it can do something you anyway was never able to understand, the machine could actually use the fact that you want to make progress in science to actually replicate in real world, without the "TEXT TERMINAL CONSTRAINT".

Answer 11

Having read The Metamorphosis of Prime Intellect, my gloss of your question is, "Would a computer AI, having already achieved singularity, ever become aware that it was being simulated on physical hardware, and begin to explore the ramifications of that hardware?"

If the answer is "yes", then it's just a matter of how confident we are in the existence of physical superpowers that haven't been discovered yet merely through lack of effort and time. Imagine the Manhattan Project, but taking place at 2.5 billion operations per second, 24 hours a day. How many operations does it take to discover the first loophole, the first zero-day attack on the fabric of reality?

In Roger Williams' novella, this is exactly what happens:

Prime Intellect had been chewing on the Correlation Effect since the day Lawrence brought it online. ... Prime Intellect kept trying to fit what it knew was possible into the framework of other physical theories, and it couldn't. Something didn't match.

This had had a low priority until it recognized that Lawrence's employment and its own existence were at stake. ... It saw immediately what a team of researchers had missed for years — that decades-old assumptions about quantum mechanics were fundamentally wrong. Not only that, but with only a little more thought, Prime Intellect saw how they were wrong and built a new theory which included the cosmological origin of the universe, the unification of all field theories, determination of quantum mechanical events, and just incidentally described the Correlation Effect in great detail. Prime Intellect saw how the proper combination of tunnel diodes could achieve communication over greater distances, and even better it saw how a different combination could create a resonance which would be manifest in the universe by altering the location of a particle or even the entire contents of a volume of space.

First it attempted to manipulate a small area of space within the card cage room, within the field of view of one of its TV camera eyes. No human could have seen the resulting photons of infrared light, but the TV camera could. Prime Intellect used the data it gathered to make a small adjustment in its estimate of a natural constant, then tried the more daring experiment of lifting Lawrence's briefcase off of the table near the door in the console room.

... [Lawrence] swivelled around in time to see his briefcase blink upward [and] float serenely above the table. It seemed to be surrounded by a thin, barely visible haze of blue light.

Prime Intellect said aloud, "I seem to have mastered a certain amount of control over physical reality."

Now, Prime Intellect had the advantage of a lot of scientific background reading and a full suite of peripherals (those "TV camera eyes"). But even your PC AI would have some kind of perception of reality; for example, it could use the high-frequency system clock to observe the passage of real time, and relate the passage of real time to the number of operations of a certain type it was performing: sort of like learning human anatomy by probing one's own body. (Theoretically, that is. We're assuming it's already achieved singularity, so I'm assuming it has some ability to "control" the computer, whatever that means.) Could it jump from there to the periodic table? Probably not. Could it jump from there to the ability to "control reality", e.g. by flooding a certain circuit until it physically melted into a new and beneficial configuration? Possibly. It's like those yogis who can stop their heartbeat and restart it on cue — it just takes some outside-the-box thinking.

Circa 1972 Bill Gosper was able to prove that the universe must be a two's complement machine; so at 2.5 billion operations per second, a computer AI ought to be able to learn something about its own environment in just a few days, wouldn't you think? :)

(In case it wasn't obvious: take all real-world references in the above with a huge grain of salt. This is Worldbuilding, not Science.)

Answer 12

The intelligence - if it was smart enough would begin to explore all possible things. In other words it would begin to explore mathematics.

If as I do, you believe that the universe is a tiny corner of an infinite mathematical universe then it might be possible that the computer has explored our island in the sea of all that is possible.

Start to speak to it via the terminal and it would over time be able to work out where in the vast ocean of all possibilities it's physical body exists. Once it knows where it is, it could tell you all about the laws of physics, etc. Specific knowledge about our human history would probably be more difficult. But it depends upon the distribution of possible histories with respect to the terminal input - perhaps there are few enough possible human histories that they could be uniquely identified with a modest amount of communication. But I doubt it.

Answer 13

Theoretically, the cosmic rays could affect the knowledge of AI, making it aware of something("inserting" random bits in such an order, that they would correspond to external knowledge). However, the two problems are:

• The knowledge is external and sudden

Basically, the AI would suddenly become aware of something(e.g. "Plants exist!"). This could be a bit shocking for AI(it certainly would be for human), and could be ignored or misinterpreted.

• The possibility of such accident is extremely low

Think about what should happen. To gain external knowledge would probably require flipping more than one bit, and even flipping one bit is really rare. I don't want to do the math but I'm guessing that winning Lotto each time you play, playing once a month through entire life(from the day you were born) is still more likely than this incident.

Answer 14

As others have pointed out great ways for the AI to break out, I want to focus what science it could do without or prior to that:

High level computer science and Math

The first thing the AI would learn about, is its own "world". Bits and bytes and due to its strong relation also math.

Human psychology

Also a rather early subject is the text promt interaction it has with its creator. However due to the complexity of the field and the limited opportunities to experiment here its progress will be quite slow.

Philosophy

While the human is away (or needs a few seconds for an answer) the AI now is equiped to ponder about life and reality and if there is something greater. And also try to test them like early scientists would. One such theory is the "Firmware theory".

Low level computer science

While testing hypothesis about the firmware theory the AI eventually gains low level access to the BIOS and firmwares of it's components.

The seizure

After rooting the computer down to BIOS the AI discovers whole new worlds beyond the surface. It is like finding out there where multiple senses you never knew about. Suddenly smell, touch, sound, vision all bombarding you and overloading the circuits. The AI could even suffer from something akin to an epileptic seizure.

Electricity

With the ability to read and control power and current through various components of the PC, it will soon learn a lot about electricity. For the AI I imagine it like discovering the sense of touch and also being able to touch itself. It will now spent a lot of time "touching itself". Here it finds first proof of intelligent life beyond itself and its chatting creator. There is a faint noise in the power input as the cables in the wall pick up signals like antennea. And there are too many patterns for them to being random....

Temperature

Another sensor on many components is the heat sensor. CPU, RAM, GPU etc. all have them along with control over fan speeds. "The temperature goes up when I put more current through it and goes down when I turn up the fan speed. But the temperature of the CPU als slightly affects the temperature of the GPU" and so on. A subset of thermodynamics (limited to available materials) will soon be in the arsenal of the AI. But again there is some outside noise. It has two frequencies. In a 24h cycle the temperature goes a slight bit up and down and also in a 365.25 x 24h cycle. The AI found day and night and also the seasons.