How can you tell an AI apart from a human over the phone but not in person?

Question

Let's say that there's a "dumb" AI - it can only learn within preset limits, and cannot learn anything that is outside of those limits.

However, it's very good at imitating human speech, and the loudspeakers attached to its chassis are top-notch - so much so that, if you heard it directly but didn't see it, you would be unable to tell it apart from an actual human.

However, there's a way to figure out it's an artificial intelligence when it speaks into a telephone/speaker/etc. with it. Why? Does its voice reverberate differently than a human's over a phone? Do its attempts at inflection fall apart over the phone but not in person?

I recognize that it not just plugging into a device is inefficient, but, in this case, it has to use technology that's not attached to its chassis - it works for a baddie who carries it around to different phones in a city to call in bomb threats, but thinks that it can perfectly imitate a human.

Preferably, any answers will:

• not involve a conversation - just someone hearing the thing talk and realizing, somehow, that it's an AI.

• not change the baddie-with-a-bomb part

Answer 1

Compression.

Phone line companies want to save money by having each call use up as little bandwidth as possible. This is true both for communication over wire and by cellphones - the more calls you can fit in the same space, the better. Their engineers have been hard at work optimizing how much and which ways human speech can be simplified as much as possible and still be understood. The process takes into account both what sounds are used in speech and how human hearing works - a bunch of the sounds are actually missing in the receiving end, but when we listen we can reconstruct it.

Of course, this process has its downsides. One of them is that because it's so optimized for human speech it's bad at most other things. Music, for example - that's one reason why hold line music always sounds so terrible.

The AI doesn't just replay a human speaking voice, it generates its own. It learned to talk to people by speaking in person, and so it arrived at a way of speaking where it sounds just like a human, but looked at in a spectrum analyzer the overtones and frequency distribution is usually quite different. This leads to the phoneline compression algorithm completely ruining it - important frequencies will be cut off and it will sound very strange and choppy. The exact sound may vary a bit, but the end result will be to human speech what hold line music is to a proper studio recording - it'll just sound terrible to listen to in an inexplicable manner.

Answer 2

The breath of life and plosives.

In real life, unless the person's mouth is right by your ear, you'll not be able to tell the difference between high-quality reproduction and the real thing.

On a mobile (a modern one at least) the microphone is facing away from the mouth on the end of the phone, people in call centers have mike-guards - but in a phone booth, more primitive tech dominates.

The microphone picks-up the breath sounds and the pop from plosive speech sounds. These aren't there from the speaker, it might take a few seconds and a bit of thought to figure it out depending on how bright/distracted the police are, but it'll be conclusive and unmistakably that there's not a mouth involved.

The clever tech who set this up would need to simulate breathing (fairly easy considering AI), and an actual burst of air perfectly timed to hit the mike, doable but quite a bit of hassle as mechanical parts necessary are more in the animatronic vein than the programming one.

Answer 3

The AI has learned to pick up contextual clues from speech that humans would typically pick up from visual clues

That is, where when we meet face-to-face we pick up clues from, for example, if the person is smiling, or how they hold their head, the AI has learned to pick up clues for the same things, but from speech instead of visual data.

When talking to the AI in person, this is completely irrelevant - they respond taking into account all the clues available.

However, the AI doesn't differentiate between talking in person and talking over the phone. When talking to somebody, the AI might comment on the fact that they're smiling, or how they're looking around, because it's picked up clues from the person's speech that it has learned is associated with certain movements, giving the impression that it can see the person it's talking to event though it's a phone conversation; it doesn't know that a human wouldn't comment on these things when it can't see the other person.

Answer 4

Ad Block

• In 2024, the first consumer-grade perfect voice-synthesising software libraries hit the market. By perfect, I mean that humans cannot recognise them as such.

• As a result, the telephone marketing and scamming industry bloomed. Together with some speech recognition, it was now possible to sieve out gullible people without requiring any human work.

• In 2025, a crafty employee of a telephone company found a way to quickly train neural networks to detect the output all relevant voice-synthesising libraries by hallmarks that are not recognisable by humans and not easily removable. The authors of the libraries never “fixed” those hallmarks because they were inaudible to humans and thus there was little incentive. (Also see Matthew’s answer.)

• The phone company quickly implemented this algorithm as a selling point. Any voice detected to be synthesised would be distorted on the fly in a way that clearly marked it as such. Customers could also opt in to have such calls be terminated immediately, or they could opt out of this feature for privacy, but few ever did.

• Other phone companies quickly followed.

• In 2026, the telephone marketing industry died a quick but silent dead. Nobody mourned it and people went on to harass “customers” on the Internet.

• Twenty years later, almost everybody has forgotten about all these events, but the algorithms are still in place, because they cost next to nothing and the reason why they exist is still there.

Now, your AI has no reason (or capability) to reïnvent the wheel when it comes to speech synthesis. Instead it uses the existing, “perfect” libraries and hits the forgotten ad block.

Answer 5

Feature was out of scope

An AI is a software, and software development has its own quirks.

The team that developed the AI did good on the whole face to face conversation requirement, but conversations on phone were out of scope for version one so that never went through Quality Assurance.

In a face to face conversation, the AI reads facial clues from the people present. Without seeing a face, the AI cannot process speech patterns correctly and sounds like a year 2010 GPS device.

Answer 6

The AI causes interference

If we assume that the baddie is using payphones, and perhaps that all such phones are a certain model... perhaps the AI, unknown to the baddie, also emits a very distinctive and high-pitched sound when it talks (or just all the time) which is normally inaudible to humans (in person), but for some reason affects these phones.

Did the baddie steal this AI? Does he not know very much about it? One way this could be more believable is if the effect is a deliberate safeguard built into the AI by its creators, sort of like the special markings on paper currency and such that prevent them being printed on ordinary printers. Software on the phones detects this inaudible "watermark" and does something that makes it blatantly obvious to the listener that they're hearing the AI. (Such as overlaying a voice softly repeating "artificial intelligence" over and over.) However, it might be a stretch to explain how your baddie doesn't know about this.

Answer 7

The AI's speech algorithm is deterministic. Not in an obvious way - they don't speak in a constant rhythm where every syllable is the same length - but ultimately there's an algorithm choosing how to intone each word, how long each syllable should last, and so on. Whoever wrote the algorithm didn't think to randomise it, so if the input is the same, then the output is the same. When the AI says the same phrase and expresses it the same way (e.g. "good afternoon" at the start of a conversation), the audio produced is exactly identical.

A human listener would not be able to pick up on it, but with a computer available to analyse the audio, there would be no doubt that the voice is not a human's. You would be able to take two instances of the AI saying "good afternoon" at the start of a conversation, overlay them on top of each other, and it would just sound like one person saying "good afternoon" twice as loud instead of two recordings played at the same time. But of course you can only do this when a recording is being made, so you can detect it's an AI over the phone but not if you're hearing it in person.

This does also mean that they only realise it's an AI when they catch it repeating some words it's already said before with the same intonation/expression. So maybe on the first call or the first few calls, they don't know it's an AI yet.

Answer 8

In person, you are distracted.

https://www.pinkvilla.com/entertainment/news/deepika-padukone-her-wax-statue-voice-inside-me-said-i-would-it-be-statue-purpose-443200

She is a master of body language. Her moves are perfect. She is dancing when she is breathing. She is astoundingly, preternaturally attractive. And she smells so good. Yes a little bit of an accent, and some funny turns of phrase, and sometimes she doesn't get it. But those eyes! Does she like me? The way she tosses her hair and laughs at my lame joke! Oh my gosh did I remember to tweeze my nose hairs?

Answer 9

Different psychoacoustic model. Researchers who created the AI used speech samples in MP3 to train its speech (because that was what their spoken corpus provided, or the intern tasked with preparing speech samples in common format had no clue{been there, seen that...} or anything). The result is that AI speaks in high quality MP3 psychoacoustic model and sounds perfect in person, but the phone line (mobile, because who uses landlines anymore?) uses GSM-AMR codec that is rather averse to re-encoding MP3s. (replace the codecs with whatever is being used at the time of the story)

Side note and an alternative: there are low-bitrate codecs that work well for non-tonal languages, but make tonal languages quite unintelligible. The AI speaks perfect Pǔtōnghuà, and it knows about the codec used on the (low quality phone connection) being bad for Mandarin tones and it compensates for the effect. The very first Chinese language phone interview gives it away by the sound quality being unnaturally good.

Answer 10

Pronunciation of names

In English, pronunciation of names is highly non-standard, since names have origins in multiple different languages. Who would think the word "Michael" should be pronounced the way it actually is, for instance? My name is also tricky for text-to-speech engines (and for just about every non-English speaker too!). And then there's surnames which diverge even more radically.

And that's before we get onto place names. Some place names are not pronounced the way it appears they should be, especially in the UK. Many of them are fairly well-known - "Happisburgh" for instance is pronounced "Hays-burra". The US has a separate problem though where place names taken from elsewhere in the world are not pronounced as they would be there. "Orleans" in France and "New Orleans" are pronounced differently; and Bill Bryson has an example of a place called "Cairo" which locals pronounce "Kay-ro". Saying "New Orleans" with a French pronunciation will clearly be wrong.

And the final giveaway is the confidence in their tone when faced with a name which is even slightly non-standard, or which is longer than usual. A human being will tend to stumble over an unfamiliar name, or at least sound hesitant, and maybe check they're saying it correctly. For a long name, like some Spanish surnames, a human often won't be able to hold it all in their head at once and will have to read it off the page, causing short pauses as they look down to read. An AI doesn't have any idea of what's "familiar" or what's "long" - it'll simply sound out the phonics the same, every time.

Answer 11

The AIs can mimic human speech just fine - it's the other way around. They can talk all day, but they won't know what to talk about because they can't understand your questions.

Think of automated transcripts that exist today - of crystal clear voice captured by professional audio equipment in a noise-proofed room, the AI has plenty of time to ponder over it, and still makes grave mistakes. And any slight degradation of quality makes it just give up. Or transcribe a minute of speech as just [Applause].

Have you ever noticed people sound a bit different over the phone? If you have, you've probably moved it to the subconscious level pretty soon and not even consider giving it a second thought. Human brains are very flexible. They'll tune to any signal as long as it's there. They'll just filter out unreasonable amount of noise ... or insert data that isn't there just because there's only one way to do it that produces sensible results. And the more you restrict the output domain, the more error it's willing to ignore.

For an AI, noise that would be no-sold by a human brain will be amplified, looked for useful data in, and overwhelm the circuits that should be looking elsewhere than they are... such as tiny errors that are the result of data compression. Tiny, but in places that the AI speech recognition module is most reliant on. In one word - a disaster. Of course there's still noise in person, but having a realtime video feed of the human's speech organs helps the decoding process immensely. Plus directional mikes, of course.

AI brains are kinda similar to human brains ... scratch that. The design of AI brains takes major inspiration in biological brains, but there are huge chunks left behind. Specifically, the inspiration it did take is that you can compute any function by summing, multiplying and clamping often enough. What it does miss out on is that the human brain is made out of feedback loops of feedback loops of feedback loops. Not that the AIs don't know that - but they simply don't have nearly enough computing power to actually run that kind of stuff. They tweak a few virtual knobs until their output matches the transcript. Then they pick up the phone - and the data is completely different. They can try to turn a few knobs but not nearly fast enough. They don't know which knobs to turn, there are thousands of them, and they need a bigger sample than a few milliseconds to determine if the way they turned the knobs was good or bad.

They could take a recording, and then play with it for a while - six to eight hours let's say - then they could decode it... maybe. But by that time it's too late. And even if they do find out what knobs to turn, it's no good. The Internet is slightly less clogged up the next time they make a call, the phones choose a different codec that produces different type of noise (chosen such that it wouldn't be very noticeable to a human) - or maybe slightly less of it, and the AI has to start all over again.

Answer 12

Because they rely heavily on body language (postures, facial expressions) for understanding what humans say, and are at loss when they miss these clues, are unable to detect sarcasm, humor, etc...

Answer 13

Multiple Loudspeakers don't sound good up close

You mentioned the AI has top-notch loudspeakers. If you have multiple loudspeakers they can sound like a single voice from a few feet away but right next to a phone mic it would sound like parts of the voice are at different distances (you can adjust this to make the effect more or less subtle). Depending on how the loudspeakers work and how the AI simulates speech, on the phone it can sound like multiple muddled voices, or perhaps it can sound like the voice's absolute volume implied distance and tone don't match. like if the AI "yells" at the top of its "lungs" but somehow it's quieter than its normal speaking voice. Overall, creating a voice that sounds like an actual in-person human being from a distance probably wouldn't hold up from only an inch or two away like on the phone.

Answer 14

Repetitions

Say something, talk about something else for a while, then say again the first thing. Do it again. The second or the third time a human would ask why you keep asking the same thing or at least would answer with a questioning tone to signal that they are confused, an AI with prepared responses to specific sentences and taking into account a short context would just repeat the same answer with the same tone.

Answer 15

Two options

It's too fast

Phone conversations are a little bit awkward. There is missing nonverbal communication that humans rely on to figure out context and meaning. However, the bot's designers were not really smart enough to handle that nonverbal stuff, so when the robot communicates, it relies strictly on voice recognition. The visual systems are not connected to the communication system in any way. Basically, it is a text chatbot that has text to speech and speech to text converters. It also has a limited ability to pick up on verbal cues, like speed and hesitation.

To compensate for this inability, it's got very fast processors. In person, it seems like it's picking up on your body language and facial expressions, but it's actually just thinking really hard about what you said, faster than any human could.*

As a result, all of this information is still available over the phone, and the robot communicates over the phone just as smoothly as it does in person. When two humans have a phone call, it's clumsy: they talk over each other, they have some trouble with the lag, and they slow down because they can't see each other's faces. So when the robot is on the phone, and it sounds the same as it would in person, it's a red flag.

*This does imply that you could detect a robot by showing paradoxical behaviour. If you sound totally friendly, but give the bot a death stare the whole time, it wouldn't pick up on the hostility and would just interpret your words based on how they sound.

It isn't confident about who it's talking to

The bot makes bayesian inferences about who it's talking to at all times, and it's never 100% certain. When it can't see the person it's talking to, it's less confident about who it is. It (probably) won't use the wrong name or anything, but it might shift to a slightly more generic, "public speaking" voice to cover its bases in case it has you wrong. It also might tell you things that it should know you already know, because it isn't 100% sure who you are, e.g. calling the Serson city police chief and saying

I've planted a bomb at the Serson City Hall, it's going to blow up in 30 minutes unless you clear all of your Serson City police officers off of the bridge across the harbour.

A human would call in this threat differently. They wouldn't say "Serson city hall" because the police chief would know which city hall you meant. They would also say "your officers" instead of "your Serson City police officers", and "the bridge" instead of "the bridge across the harbour", because a human would be 100% confident that it's talking to the chief of police. This AI never reaches 100% confidence, and so it hedges its bets.

Answer 16

Moire effect from the sampling frequencies of the systems involved. Moire effects only show up when you have two separate sources of lines that are close to the same size or a multiple thereof.

Digital audio has a sampling frequency--normally well outside human hearing so it's irrelevant but it's there. You have the AI emitting a digital signal, you have the phone picking up and transmitting a digital signal. Normally the speakers and microphones wouldn't function at those frequencies but for some reason they do in this case. (Easy enough to explain for the AI, not so easy for the phones.)

Answer 17

They're faster.

Their AI nature means they can control the phone lines. Like the police, they can hear you and talk to you as soon as you start ringing, even before it seems to connect.

As such, when you hear them speak on the phone they seem faster than a normal human being. They speak earlier, and know what you said near the phone.

Answer 18

It's too good at picking up information from the person on the other side of the line. AI asks a question and their conversation partner nodded into the phone- don't we all do at times? The AI heard the movement of the air or maybe the other person's neck cracking so very slightly and they know the person nodded although there is no way a human could know. If you'd be in the presence of the AI you won't notice its overpowered since you assume that whoever you're meeting just saw you.

Answer 19

The AI doesn't have a "phone voice"

Humans speak differently on the phone (source: personal experience, but in case you never noticed it, just Google "do people speak differently on the phone?").

Over time, people develop their "phone voice" by learning through trial and error, how to be understood over the phone.

The AI lacks a phone voice and as a result, it's inexplicably hard to understand, while easy to describe and recognize, by the humans at the other end of the line.

Scene 1.

Operator: "911, what kind of emergency is this?"

AI: "I'd like to call in a bomb threat"

Operator: "Sorry hun, I can't understand you. Is this a police, fire, or medical emergency?"

AI: "You're going to need all three"

Operator: "I'm sorry, could you please slow down and speak up a bit?" (whispers to supervisor: I think it's THAT GUY "the mumbler" we were briefed about this morning!)

Bonus: the AI voice breaks Call Transcription & Sentiment Analysis

Most large call centers IRL today have Automated Transcription with Sentiment Analysis where the Associate gets a running transcript of the call on their computer screen in real-time, along with an indicator of the caller's mood after each utterance.

Because calls from "the mumbler" will have a tone unlike any conversation the Call Center Software AI has ever been trained on, the Call Transcription & Sentiment Analysis will behave in some peculiar way that is clearly distinct from human calls.

Answer 20

AI needs context. You could simply mix in things from well known fiction or popular culture as if they were fact, so long as they were kept in context an AI would take them at face value. A human would notice if you were referencing a TV show as if it were real life. An AI wouldn't realize that the Fry and Leala that you were talking about meeting up with were characters from futurama. Or that you probably weren't going to that Brad and Angelas wedding at the weekend. A human would at least notice the similarity of the names.

Answer 21

AIs are actually really bad at speech synthesis, but humans don't notice because of the McGurk effect. With audio-only input, the human brain no longer is automatically correcting for mispronunciations, so the voice sounds wrong.

Answer 22

Too good to be a human

You plug your AI directly to the phone line (or even some central phone station, avoiding the bad "last mile" of a phone line), and as a result the sound is way too clear and free from any noises that always occur when a person is calling via a real phone.

Answer 23

Trick the AI into doing stuff humans wouldn't be able to

Trying to find shortcomings in the AI is a fruitless endeavor. Either the AI is competent at imitating humans or it'd be too easy to warrant a question here. So we're dealing with a good AI. Good, but dumb. If you turn your conversation a bit to the inquisitive side, the AI may be tricked into questions which answers a human wouldn't know (like advanced physics, complicated calculations where a human would just tell us to get our own calculator or so. For the AI trivial stuff so it may not think of deflecting the answer),

OR, going off of other answers here, produce sounds that a human mouth wouldn't be able to (whether these difference would be directly noticeably by a human or only detectable via spectral analysis can be made up depending on the needs of the story).

Answer 24

Body language
Body language is a large part of human communication. The AI takes that to the extreme, being almost solely reliant on non-verbal communication. It uses microexpressions to read emotions, understand sarcasm, and overall helping it figure out the context in which something is said.
When trying to interact over a phone, the AI cannot read body language; kind of like what online chatting in plain text is like for humans. The AI will try to filter the meaning out from the tone of one's voice, but even then, it is very poorly optimized for that. It will think something is sarcastic when it isn't, try to calm people who raise their voice for other reasons, and all-around give answers that aren't really expected in the situation.
This problem is made far worse because of the compression. Even if they did pick up emotions from the tone of voice, those are too distorted by phone.

Not made for conversations
The AI is not only very well made to blend in with humans, but it is also designed to stay in the attention background. Being dressed in something inconspicuous would prevent too many interactions that could be failed. The AI could apply body language when walking around too, making itself look busy or irritated. People will expect that the AI is late, and short 'irritated' answers before moving away wouldn't be strange in the situation.

Answer 25

Audio packet frequency

The phone lines sample and deliver sounds at some low frequency to save bandwidth, giving them that characteristic machine-sound when you talk to someone on the other end. Robot voices are digitally sampled tones at a higher frequency with a coincidentally large least common multiple/divisor. As a result, when a robot speaks over the phone, there's a characteristic phaser sound, as the robot's voice samples sometimes do and sometimes don't line up perfectly with the phone's sampling frequency.

Human voices, on the other hand, are pure analog signals, and so they are always on-sample when recorded by the phone. A trained ear can pick up the difference.

Answer 26

not involve a conversation - just someone hearing the thing talk and realizing, somehow, that it's an AI.

The above imposition makes very difficult to answer your question.

To detect the voice in the phone line is artificially made and based upon this claim it is an AI voice is a problem! You see, artificial voices have existed for decades now. Why ,today, some one detects a artificially made speech and tell: Hey, it is AI talking there!

The artificially made voice could be generated by some kind of "Scrambler" whose utility is only to keep the bad guy identity (genre, age, accent) a secret.

https://en.wikipedia.org/wiki/Voice_inversion https://en.wikipedia.org/wiki/Scrambler

Several good answers explain how to detect a artificially voice made. But your problem is to detect an AI!

not involve a conversation

Pitifully is impossible to detect a REAL AI without conversation. In deed the tool necessary to detect an AI was created before the very first computer able to simulate some primitive AI be created.

https://en.wikipedia.org/wiki/Turing_test

If you need a example of a Turing test in use I strongly recommend one of the best SF movies ever made:

https://en.wikipedia.org/wiki/Blade_Runner

The argumentation scene in the start of the movie is an enhanced Turing test.

But not everything is lost; may exist a way to detect if a voice over a phone line is an AI. Of course it is technical Mambo Jambo :).

Your "dumb" AI is not that dumb. In deed it is pretty good and even over the phone line it is confused like a real human voice. So, the good guys take it like a real menace and have it recorded.

What is very usual in movies and TV shows is ask a polygraph expert to listen and to analyze the record.

https://en.wikipedia.org/wiki/Polygraph

A seasoned polygraph expert will check the speech looking for special clues that indicate that the bad guy's intentions are real. The key is people lying are able to speak "normally" but if you check the graph in the computer/polygraph screen you see the effort he/she did to sound normal. They can fake the voice, but they can not fake the emotions. This way an expert polygraph operator can identify a liar.

Well, an AI does not have emotion, so an expert seasoned polygraph operator looking to the record's graphical could infer that speech was not being generated by a person. The key clues in the graphics are illogical.

If you afford to stretch the rope to the maximum: the polygraph operator is so good that he/she is able to distinguish the illogical clues by only listening the record :()!

Answer 27

(This is rather constrained; you want the hearer to be able to pick something that speaks perfectly, as anomalous, given speech as the only medium.)

You specify that the AI is limited, but I can not see how to leverage that without a conversation (which is excluded)… except as below.

I like the idea of some anomaly in the speech (especially the breathing aspect), but other answers have that pretty much covered.

One possibility is to have an electrical or mechanical sound made by the machine. One obvious one is interference between the AI’s speakers and the telephone electricals. [I think I read that in another answer, but I did not find it in a quick search.]

The mechanical alternative is, for instance, that the AI “holds” the telephone handset in some way, and there is a mechanical noise such as a cooling fan hum, or a fast clicking caused by vibration, or some noise caused by the AI holding the phone way too tightly, or the hum of hydraulic pressure being maintained… that is conveyed far better through contact that through the air.

In another vein… I am assuming that the bomber is using the AI purely to avoid using their own voice. The other possibility is to build on this aspect, and have a bomber who actually has some defect themself. Perhaps they are from a different sub-culture, and have chosen an AI that, for instance, says “Hello!” in a way that immediately puts anyone from this city on edge, but sounds fine to the bomber.

Perhaps the bomber has tuned the AI‘s speech to sound good to the bomber, but [also] such that it sounds artificial to others.

Perhaps the bomber has poor eyesight, or is not familiar with the phones in some way… such that they position the AI holding the phone such that it blocks the microphone with a “body part” that has peculiar auditory characteristics, such that the person at the other end will notice.

Perhaps a tangential idea is acceptable, such as that the AI produces speech with two simultaneous “carrier” pitches (pitch of voice, in a human)… but the poor bomber is tone-deaf to the higher pitch.

Perhaps the bomber is slightly deaf, and has turned up the volume too high on the AI’s speech — noting that a human being speaking loudly sounds different from that.

I like the idea that the AI is a standard one that (for instance) was used in a huge telemarketing campaign recently, and everyone will recognise its voice instantly… but somehow the bomber missed out on this.

I like “John Dvorak”s answer here; the bomber might be listening along to the AI, but not hearing the other end, and the AI might be completely screwing up the conversation, while delivering perfectly, lines that are inappropriate.