In this scenario an a tiny device with artificial intelligece was left alone with tons of data about physics and media, and its main directrix is to protect a human, and for doing this it is capable of self-engineering and changing both hardware and software.

Along the time, the AI can became more intelligent and powerful at the level of being capable of turn itself in a weapon and still be faithfull to its main directrix?

So, at some point it became self-conscious but still not having feelings but follow the ethics.

  • 1
    $\begingroup$ Define "feelings" and "self-conscious" I ask because if you're going to say "can I have A but not have B," defining the exact bounds of A and B will matter quite a lot. Also, have you heard of p-zombies? They're really useful for exploring questions like these. $\endgroup$
    – Cort Ammon
    Aug 11, 2017 at 2:39
  • $\begingroup$ Feelings in this case would be just the behavior of a feeling creature, not necessarily the sentience itself, because defining sentience is quite hard. But since the AI is exposed to lots of data about it, it can find useful and try mimic this. $\endgroup$ Aug 11, 2017 at 2:53
  • 1
    $\begingroup$ You see why those definitions start to be important. The only known example of sentience that everyone agrees upon is humans. Everyone agrees humans are feeling creatures. That means we have no examples of what you are looking for, so we're dependent on definitions. I think you could definitely write a book without those definitions, but in a Q&A format, it gets really hard without some semblance of agreement of what a non-feeling creature behaves like. $\endgroup$
    – Cort Ammon
    Aug 11, 2017 at 3:00
  • $\begingroup$ Humans have ethics, but are not bound to it. At least not 100% of them. I strongly feel the answer to your question is primarily opinion based. $\endgroup$
    – L.Dutch
    Aug 11, 2017 at 5:12
  • $\begingroup$ There are very intelligent humans without feelings or ethics. So feelings and ethics have nothing to do with intelligence. $\endgroup$ Aug 11, 2017 at 15:48

4 Answers 4


Yes, but it depends greatly on how you define "self-conscious" and "intelligent". As a professor researching AI, we need "working definitions" (meaning not universally accepted by all) of these terms. We call them "working" definitions so that we can work with them and know when we have achieved them and what we mean when we talk about them.


I regard Intelligence is an ability to internally use "models" and "simulations" to figure out a coherent picture of what is not self-evident or instantly visible. That may be figuring out something historical (tectonic plates used to be together in Pangaea; or a victim was choked to death), something in the present ("this patient has tuberculosis", or "that column of smoke can only be due to a house or car fire at the base of it"), or something in the future ("This pit is disguised well enough that deer will not detect it", or in an unfamiliar grocery store, "the milk is probably in that refrigeration section at the back left").

You do this with mental models, which are an abstraction of rules and facts you have learned. They can be different from person to person because of what they learned and how; but in general we all have a model of how grocery stores work, and some basic navigation ideas about them: they have some area for fresh vegetables, and for deli, usually near the meat section; another section for dairy. The candy is not generally beside the medicine or the cat litter or the tin foil. Aisles are wide enough for carts. There are carts available!

Part of the model operation is able to identify "hidden" likelihoods: Suppose you are unconscious, and wake up in an empty room with a nondescript door. You don't know where you are. You walk through that door, into something that looks like a grocery store: How do you know? Because enough visual model parameters were met that the neurons used to make up your mental model of a grocery store fired; while the hundreds of other models in your brain found inconsistencies and did not fire: It isn't an office, or a car garage, or a home, or an amusement park, or a movie theater or a police station or restaurant or barber shop or movie set or space station or casino: Given the visual cues, the Model most consistent with those cues and with the fewest negative clues (things that do not belong) is that of a grocery store.

So you will conclude you are in a grocery store and because of that expect to find the other parts of a grocery store specified by your internal model, even if you cannot see them: There must be a cash register somewhere, near the front so people can leave with their groceries. There will be only one such front on one side of the building; if you walk the perimeter you should get there.

That is "intelligence", good predictions of reality (what really happened, what is really happening now, or what is really going to happen) based on clues. It does not require emotion. AI makes good predictions about what the stock market is going to do in the next milliseconds, minutes, days or weeks or years, with stunning success. It doesn't have any emotion about it; the only requirement is to be correct far more often than it is incorrect.

Remember, this is my working definition of intelligence; other researchers or laymen may have their own ideas: Mine is chosen because I think it is clear and testable and measurable, which are traits I need in order to do research that is more than guesswork and pointless circular philosophical reasoning. The same applies to the definitions below, so I will stop repeating it.

Self Consciousness

I will break this into two things; self awareness and consciousness.

Self Awareness

This is having an internal model of yourself. It is an abstraction, don't forget. But it means you have an ability to predict what you will like, what you find boring or exciting, what you find frightening, or pleasurable, or what angers you. You can predict the physical feats you are capable of doing, or not doing. And the mental feats: Whether you can solve differential equations, whether you can draw, whether you do well at Trivia puzzles, whether you can hold your liquor. Differences between your mental model of yourself and reality let you know when you are sick, weak, in pain, or damaged: If your left knee is not working correctly (say buckling), that is because it is not meeting your expectations built on experience of how your left knee works: Where are those expectations encoded? What neurons are firing to tell the rest of your brain something un-expected transpired (or failed to transpire)? In your mental model of your body and how it works, which your brain has been learning (and updating) throughout your life. (It is updated because bodies change gradually over time, and sometimes drastically through amputation, spinal cord injury, surgery, or other disabling accidents).

Self Awareness is having a model of yourself in the world that can be used for planning your actions in the world. You do that with mental simulations using many other models: For example, "If I go to the game, it will be fun. I won't get in without a ticket; I must buy a ticket. I could walk, but it is too far, so I will need a ride, so I will need to arrange for Uber. I must be sure to bring my cell phone. Now let's see, what will I wear..." and so on. Every one of these thoughts is triggering a simulation. The fact that they are in a chain leads us into Consciousness.


The human brain has (on most recent scientific data I have read) about 150 relatively large clusters; each cluster is dense in connections; and then the clusters themselves have communicating parts. Kind of like cities and towns connected by a highway system. There are hints this pattern is fractal; meaning each cluster may well be composed of sub-clusters with their own kind of highway: Like a City can be a cluster connected to an Interstate Highway system, but the city itself has main thoroughfares and its own internal transportation system, ranging from one-lane alleys to six lane elevated highways.

The reason I say "hints" is the best measuring equipment available for this kind of work is still not exactly high-resolution; and neurons are cells, microscopically tiny.

If we look at this in terms of predictive models; the brain may be capable of many billions of such models; particularly because we use componentization: We model "a wheel" pretty simply, whether it is on a baby carriage, bicycle, motorcycle, car, truck, aircraft, tank, bulldozer, etc. For each of the more specific instances we just have additions to the idea of "wheel". The first three can all have "spoke" wheels; but we don't expect to see those on an aircraft or a tank. What we do expect to see is something round that rolls and turns on some sort of axle and is sturdy enough to support a load. The neurons that decide if something is a wheel or not a wheel or just looks like a wheel or is a broken wheel can inform many thousands of models.

So my working definition of Consciousness is pretty simple; a kind of random walk through the brain. A train of thought is these different models informing each other, endlessly. It is the communication of different parts of the brain; we get ideas, pursue them for awhile, they lead to other things and eventually decisions about what to do. The body can DO things; whether that is changing the channel or turning the spare bedroom into an art studio.

Consciousness is simply this train of thought, of one "state of mind" leading to another "state of mind". This may be goal directed; there can be an objective. I want to solve an equation, the models in my brain that do that work interpret the equation, characterize it, recall a solution path, and start trying things.

Consciousness does not require language; as many people assume. I believe internal speech is actually something that happens in the wake of thought: A model is invoked that knows what to do with second order differential equations. One of the connections in that model is its label, "Second Order Differential Equation", like a hashtag. It is after this model is activated that it activates the (internal) speech center and we (internally) "hear" the words.

Language can be useful here; those words (that hashtag) are each themselves a model that can be physically connected to several other models that then get activated too (e.g. there are many techniques for solving such equations; so they all should weigh in on how well they fit the equation in question).

This is what I think consciousness is; this flow of activations of millions of models; often with language elements in their wake, and our mental models of grammar and word choice forming actual sentences we "hear" as an internal dialogue of questions, answers, arguments, quibbles and eventually (perhaps) conclusions about what to DO.


Yes, using workable definitions, a machine can be intelligent, self aware, and conscious. This does not require emotions at all; it doesn't have to want anything, fear anything, love anything, or feel pain or joy. It can have a goal or objective; including an ongoing goal (say helping others).

It also does not have to be more intelligent than humans, or more powerful than humans. It also does not have to be insane and out to kill humans or take over the world or anything like that. Just like a human, it does not have to get trapped in "logic" puzzles that imply it should kill all humans or end all life: Like humans, it can simply reject outlandish conclusions out of hand no matter what logic led to them.

  • $\begingroup$ You should come over to Stack AI. A core purpose of that Stack is to discuss precisely these issues. $\endgroup$
    – DukeZhou
    Aug 11, 2017 at 18:09

I suspect that the answer is 'No', because human-like intelligence probably requires feelings in order to be human-like but, as Cort says, clear definitions are probably important.

Personally, I think the idea of self-awareness is essentially a matter of emotion and feeling: it is necessary for 'I' to exist in contrast to the 'other' and that requires a definition of what 'I' is, as well as an appreciation of why that should be important, and that becomes a matter of physical feeling and emotion, I think.


Actual feelings

I would try to approach the question from a bit different aspect than the one in the comments.

Let's start with exactly human-like AI and taking emotions away from it.

We already have in our society people described as borderline or emotionally numb. While it is categorized as a disease and creates many problems with interactions with the rest of the society, it shouldn't be that much of a problem for the AI.

With AI being sentient and self-conscious it should be able to understand the purpose of its creation and the reasons for the boundaries humans gave it. What's more, equipping itself with emotions would probably seem as a disadvantage as emotions often take power over reason and may cause much harm.


If you want to talk about simulating the behavior of felling creatures, it is definitely possible. Probably, it could be even achieved with current state of technology. Feeding a neural network with enough data and providing the computer power needed should give us results close to actual human behavior. However, it would probably do all the things that a generic human would do - behaving rather normally.

Creating an actual personality would be much harder, as it would have to mimic a certain person. What sounds reasonable is a mix of those - feeding the AI the data about people with similar personalities to create an AI that would fit that type. However, it would still be far, from actual feelings or emotions.


Human-like intelligence is of course theoretically possible for AI.

The real question to answer in this case is: would the AI still be required to learn by experience or could experience simply be uploaded?

If the answer is "AI must learn through experience", then likely an emotionless, sentient AI would be very ineffecient at survival until an appropriate amount of wisdom/experience was obtained. Emotions are a key component of human survival, particularly during youth.

If the answer is "AI may obtain wisdom via digital information transfer", then emotionless AI might have a real shot.


Not the answer you're looking for? Browse other questions tagged .