21
$\begingroup$

How can an authority verify the identity of an AI?

An easy to explain example is the equivalent of a drivers’ license. Consider a typical situation, in the near-ish future: Police cruiser pulls over a car that has a passenger but no driver. Starts to admonish her that self-driving cars must still have a licensed driver, even on the e-lanes.

The car replies that it’s a prototype AGI and had the officer looked at the report when running the plate, he would see it is flagged exceptional. It (the car) holds a valid drivers’ license in the state.

The license is posted in the door frame, next to the weight and tire stickers.

Now, how can someone verify that the license is shown belongs to the entity in question? For normal licenses, we use photos of the face and descriptions of height and eye color, etc. But an AI will not have “biometric” attributes, and any such affordances it does posses will not have the same property of being fairly unique and unforgeable. All Johnny Cabs look alike.


Note that the idea here is to verify that the system has an approved use based on its “skill” and knowledge. This is not as strong as a unique identity which you need for determining property ownership for example.

Keeping with the easy-to-understand drivers’ license, the threat model would be for some person to slap a sticker on his ordinary self-driving-car that has some capability for normal driving but would not understand someone directing traffic with flags or other exceptional cases.

$\endgroup$
18
  • 1
    $\begingroup$ Question: are we concerned about purely malicious identity theft (ie my identity has been stolen) or the AGI being in on it (ie ‘passing’ the identity to another entity)? $\endgroup$
    – Joe Bloggs
    Nov 6, 2017 at 17:35
  • 4
    $\begingroup$ For now all regulations are "per car model" or "per driving system model", not per particular piece so "All Johnny Cabs look alike." is not a problem - they are entitled to ride on public roads or not. All of them (unless hacked). $\endgroup$
    – Mołot
    Nov 6, 2017 at 17:54
  • 3
    $\begingroup$ The more interesting question is why the car needs to be pulled over at all. If there is a moving violation, and the officer believes the self-driving software to be malfunctioning --which might be true--, the officer can immobilize (or even impound) the car, regardless of the AI's claimed identity and license. If the officer is mistaken, the car's logs will show the discrepancy. If the officer is abusing their authority, then it's a whole different question. $\endgroup$
    – user535733
    Nov 6, 2017 at 19:47
  • 3
    $\begingroup$ Do we think this can be reopened with the edit? I think it's an interesting question, and with the purpose of the investigation now stated in the question, I think that reasonable answers can be provided (such as the use of cryptographic signatures) $\endgroup$
    – Cort Ammon
    Nov 6, 2017 at 20:14
  • 1
    $\begingroup$ Beyond the technology of field identity verification, there are plenty of legal questions involved. Kudos for limiting the scope of the question to field-verified ID. $\endgroup$
    – user535733
    Nov 7, 2017 at 1:05

8 Answers 8

29
$\begingroup$

What you have, what you know, who you are

Those are the three ways that an unknown person is authenticated and authorized. For humans, "what you have" usually means a physical key or pass card. "What you know" means a password or passphrase. "Who you are" means unique biological data that is difficult to fake or duplicate, such as finger prints, retina, etc.

The digital equivalent of "who you are" is a hash. Hashes are a form of one way compression where a quantity of binary data is put through a hash function resulting in another, much shorter number. Hash functions are designed in such a way that if a single bit of that number changes, there will be large, easy to see differences in the resulting hash. There is only a one bit difference between 8 and 9. The below example shows the large differences that a single bit will bring.

$ echo "01234566789" > 09.txt
$ sha1sum.exe 09.txt
0230f435629ff197db2935a4f2d58d234c74de0d *09.txt

$ echo "01234566788" > 08.txt 
$ sha1sum.exe 08.txt
f2408b6a33c9752882dd46aae70674f4f0597e92 *08.txt

The requirement for these three methods to be secure, in real life and digitally, is that they are extremely difficult to duplicate. Note also, that encryption is used for verification as well as concealment. In this case, encryption will be used to ensure that the AI gestalt installed in the car is authorized to be there.

What the car has

The car has a stamped nameplate with a barcode/QR code of its serial number and certificate of authenticity. The certificate of authenticity contains two thumbprint hashes of the AI's gestalt; one by the manufacturer and one by the certifying authority. It also has a QR code of that AI's public key. In addition, the car will have a FIPS 140-3 cryptographic module for generating hashes of itself and responding to authentication claims. This crypto module will work much the same way as the chip in your debit card. Of necessity, this physical protection must extend to whatever hardware the AI is running on in the car.

What the cops/inspectors have

The cops have a big list of authorized AI gestalts hashes and associated public keys. They have the public keys associated with the private keys that signed the AI gestalts. With the hashes and public keys, they have everything they need to cryptographically verify that the AI gestalt is "who it says it is".

What the certifying authority has

They have a copy of the gestalt (though not the source code that created the gestalt), the manufacturer's hash of the gestalt and manufacturers public keys. They do not get the private keys because that only leads to shenanigans (which this whole complicated expensive process is trying to avoid).

When the certifying authority creates their hash for the gestalt, they will add a salt to the gestalt before hashing. Salting the gestalt before hashing ensures that the gestalt is different from the manufacturer's hash ('cause it doesn't make much sense if the two hashes are the same). Salting also makes it much harder for attackers to recover the certifying authorities private key because to recover the key would require recovering the salt as well as the private key. This is much much harder.

Spot check that the cars crypto module and AI brain are FIPS 140-3 compliant.

What manufacturer has

The car/AI manufacturer has the private key that they use to sign the AI gestalt before installing it in cars. They also produce a hash of the gestalt. Manufacturers go through the same process of salting and hashing the gestalt that the certifying authority does.

They also make sure that the crypto module and AI brains actually are FIPS 140-3 compliant. For each crypto module, they install a new private key.

Buzzword Compliance: Blockchain

As much as I despise buzzwords and handwaving "it'll do magic" tech, there is a valid use case for blockchain tech in this instance. Consider that blockchains are really just big distributed ledgers shared by people who really don't trust each other.

In this case, a blockchain would be built to hold the car specific public keys, the manufacturer and certifying authority public keys, and hashes of certified gestalts. The blockchain becomes the canonical record of who did what and when. This prevents attacks where someone attempts to change a public key sneakily. They can't since the record is immutable. Well, they can change it but that will require enough resources to get a majority on the blockchain. Such a majority will be noticed.

As each new gestalt is published then certified, all the appropriate information about that gestalt is added to the blockchain. Once added, it's permanent.

Authorization Verification Process

This is how things might go

  1. The cop will scan the name plate of the car for manufacturer, model, manufacturer AI gestalt hash and certifying authority AI gestalt hash.
  2. The scanner looks up the manufacturer, model and both hashes against the known-good configurations. If all four data points match, then the AI in the car is probably good to go. This is the weakest but fastest verification of the car's AI.
  3. For stronger verification, the cop will plug in the scanner to a data port on the car. Once connected, the car will respond to a command to encrypt some long really long and randomly generated number with the AI's private key. Since the scanner also has the public key associated with this car, if the crypto module returns a strong that can decrypted with the public key then the cop/scanner knows that the crypto module is genuine. This verification process prevents an attacker from stealing the correct response from authorized hardware and just replaying it back to the scanner. This, in addition to previous verification steps.
  4. The strongest verification will be to take the car into a shop and examine the crypto module. Since it's FIPS 140-3 compliant, any tampering will be evident and the car will fail inspection. This, in addition to all the previous verification steps.

Weaknesses

Since everyone messes up and even strong security systems eventually are compromised, as described, this system does not account for the need for rapid change of public-private keys should they be compromised. Say, someone at the manufacturer or certifying authority fat fingers which key to add to the block chain and they add the private key when it should have been the public key.

There are a thousand ways to break this system and it will take a lot of very careful planning in order to make it very hard, even for nation-states to compromise it. (Ha! Well, that's far far beyond the original threat model but this was fun to write.)

Modern Web Cryptography

Much of the process and crypto primitives described above can also be found in modern web crypto. If you see an 'https' then your browser is using a system of public and private keys to authenticate and authorize the server you're connecting to.

Not yet covered areas

What this process doesn't cover is the data that the AI uses to navigate with. Where that comes from and how it's used is outside the scope of this question.

Attacks on this data would preserve the integrity of the AI but cause errant behavior.

$\endgroup$
8
  • 2
    $\begingroup$ We need someone from crypto.stackexchange.com to verify this approach. $\endgroup$
    – Euphoric
    Nov 7, 2017 at 11:28
  • 2
    $\begingroup$ @euphoric not that much. From what I humbly know about crypto, the above is correct, and the question is not tagged[hard-science] so there is not much rigor required. The material in this excellent answer is more than enough for a worldbuilder to make their car AI sound realistic. EDIT: +1'd $\endgroup$ Nov 7, 2017 at 15:54
  • $\begingroup$ Would it be out of the question to use a MAC? It is both laser printed into the hardware and uneditable in software when calling. They are used as unique identifiers for all computers and use 6 pairs of hexadecimal values so that it is almost impossible to have too many (it's the SSN for computers). I understand you can spoof it, but if the cop checked the hard copy against the spoof wouldn't the charge be the same as falsifying federal documents? $\endgroup$
    – Liam
    Nov 7, 2017 at 16:25
  • $\begingroup$ @Liam your suggestion isn't as strong as a private key. A static piece of information can be faked. One would only need to discover the MAC then it can be copied to an arbitrary number of places. $\endgroup$
    – Green
    Nov 7, 2017 at 17:08
  • $\begingroup$ As @Green says, a MAC can easily be copied. A hash of some unchangeable part of the AI could only be copied by moving the entire AI to the new hardware (spoofing is theoretically possible but computationally unattainable). If the MAC is legally protected, however, then it can be copied, but there would be some legal penalty for doing so. $\endgroup$
    – Erigami
    Nov 7, 2017 at 17:21
14
$\begingroup$

This can be done in a similar manner as we check the authenticity of software today - using license keys and checksums. This way we can be sure that:

  • The program in question is legitimate, particularly if license is registered in central database;
  • The program in question was unmodified, at least to the point of its installation;

However, there are no easy ways to check if the program has been hacked. If the program in question is a "learning" AI, then every installation of it would be different, and while we can still say that the program initially was exactly what we wanted it to be, there is would be no way of telling how this program changed from that moment and whether those changes were legitimate.

If the program in question is a "static" AI, then situation is more manageable. The difficulty of this task will be comparable to checking if a particular iPhone is not lost/stolen and it had not been jailbroken.

$\endgroup$
2
  • 1
    $\begingroup$ This was my first thought as well -- even for such a fictional "learning" AI, I'd imagine that there's still core business logic that you could do an MD5 on. It would be separate from the "data" which it has learned -- kind of like looking at the DNA we're born with vs behaviors we've developed based on things we've learned. $\endgroup$
    – A C
    Nov 6, 2017 at 19:09
  • 2
    $\begingroup$ Uh, certificates. Certificates, such as those issued for code signing (which produce a signature that has to match the code, or a checksum/hash of what they sign) could easily fill the requirement of ensuring that the code hasn't been altered... that's one of the functions code signing certs perform today, after all. You'd sign the algorithms, rather than the dataset, which would at least ensure that the code is unmodified. $\endgroup$ Nov 7, 2017 at 6:45
3
$\begingroup$

Track the transfers of entity from vehicle to vehicle, then use the vehicle identification.

What you’re trying to do is authenticate the ‘mind’ of the vehicle to make sure that it’s allowed to drive. Assuming that the AI’s might happily share licence information with each other and are too complex to perform a checksum or program integrity check on the only way you can authenticate the mind is via the body, in this case the vehicle.

So: only licenced engineers are allowed to transfer AIs, whenever they do they must log the new location of the AI. At that point verification becomes a matter of physical authentication which can be done many different ways.

On a more abstract bent: what you’re aiming to do seems a bit pointless. If you’re authenticating that the AI knows how to drive you can essentially perform a roadside ‘driving test’ to check that the vehicle will respond appropriately to a set of standardised tests. The AI should be able to do the test very rapidly if appropriately designed. It doesn’t matter if it is the AI that you think it is if it knows how to do the thing you want it to do. If one of the questions is designed to weed out the prototype ‘trusted’ cars from the non trusted ones then the non trusted ones will fail the test. If the non trusted ones pass the tests then they are, to all intents and purposes, trusted.

$\endgroup$
2
  • 1
    $\begingroup$ "only licenced engineers are allowed to transfer AIs" — did it ever stop anyone? I mean, PlayStation 3 and 4 are both designed from the scratch with the idea they will be attacked, and still people found a way to run "unauthorized" software on them. Hacking tractors is also a thing. It is happening already, as farmers dislike limits imposed by manufacturers' pseudo-ai. $\endgroup$
    – Mołot
    Nov 7, 2017 at 6:42
  • $\begingroup$ @Molot: Not really stopped anyone, but then the unauthorised engineer is either moving an inferior AI into a licensed vehicle (why would you do/the superior AI let you do that?) or moving a superior AI into an unlicensed vehicle (ditto). Also people have showed they can spoof some biometric signals: That doesn't reduce the usefulness of having the security measures there. $\endgroup$
    – Joe Bloggs
    Nov 7, 2017 at 7:18
1
$\begingroup$

You cannot do it, unless you make some assumptions

Making the full proof cannot be done here, because it requires a university level course in Information Theory, so I will just state it as bare fact:

You cannot reliable authenticate a piece of software when someone else is in control of the hardware it resides on

This is — for instance — why we cannot have cheat-proof computer games. So unless you make certain assumptions, the answer to your question is:

That which you ask for cannot be achieved

And for the record: the same goes for bio-metric identification of humans. Unless you make some assumptions, then you cannot authenticate a human against their driver's license with 100.000% certainty. Instead you must make some assumptions, such as "Human fingerprints are unique and can always be distinguished from each other".

The assumptions when it comes to AI

The assumptions you have to make in your case is:

  • A piece of hardware that has been validated by the enforcement agency (i.e. Police/Highway Patrol) can make an accurate memory dump of the AI in question, being given the code, the whole code and nothing but the code.
  • The AI cannot be reprogrammed / replaced.

If this is the case, then it is actually quite easy. The officer will make a dump of the AI's software and the officer's hardware unit will make one or several hashes of it. These hashes act as fingerprints of the AI. In fact they can even work as the identity of the AI.

All the officer then needs to do it make a lookup and verify that the AI with these hashes is authorized to travel in / drive that kind of vehicle, in that area, on those types of roads, under those conditions, and so on.

Alternatively the AI comes with an public authority issued digital certificate that lists all the rights of the AI. The certificate will certify that the AI with the hashes specified has the following enumerated rights. The officer's hardware needs then only verify the certificate, which is a mundane affair.

This then comes with an additional assumption:

  • Digital certificates cannot be forged

The weakness is of course that 2 out of 3 of those assumptions are fairly flimsy. There is nothing to say for instance that the original AI was not disconnected from the actuators of the car, and another AI installed in parallel with this one that does the actual driving. But when the officer connects their scanner to the car, it is the original AI that responds.

There is no way around that unless the government has a monopoly on building AI cars and has 100% control over the hardware. And even then you have the issue that no-one will ever have 100% control over the hardware...

$\endgroup$
1
$\begingroup$

I'm not a security expert, but I think you can do most of this with simple private key/public key encryption. You just need to establish the correct protocols so everything can be verified, and trust the private keys remain private. So it's not infallible, but then again nothing is.

The car and cop need to be able to generate encrypted signed messages meant for the car's manufacturer, meaning no one else can decrypt the messages and they are verified as being to-and-from either the car and the manufacturer or the cop and the manufacturer.

The cop generates a random token and it sends an ID request to the manufacturer for this car's stated name. At the same time, the car uses the random token and sends it's authorization to be verified to this officer. You could do it other ways, but this way means the officer and the car both have to consent to being verified.

Since the cop is getting verified too the cop also sends the token to the main police authorization service (we'll call it Big Blue). This step isn't necessary. I just like it this way so not just anyone can verify a car.

The manufacturer decrypts both messages. The car says "Hello manufacturer X, it's me Bob, please verify with officer Pebbles Larue (token)", then it gets a message from the cop and says "This is officer Pebbles Larue, please tell me if this is Bob or not (token)," then the token gets sent to Big Blue and Big Blue says officer Pebbles is on the up and up, so the manufacturer responds to Pebbles with a thumbs up emoji, meaning this is indeed Bob.

$\endgroup$
1
$\begingroup$

Assumptions:

  • The AI is a true AI, and has operational control of its hardware.

  • It has some form of memory that is write once. This part of the memory is not accessible without destroying the machine.

Now: Part of that read only memory is the private key for Hal's private public key encryption. The AI is the only place in the world where this key is written down. To get this to be true, the AI has to be able to generate the key pair on it's own (trivial for a computer) and register the public key in a key repository.

So proof of identity works like this:

COP: Who are you HAL: I'm Hal 2764. I'm registered. I'm listening on redtooth 7

The cop's computer takes today's date, a random string token, and it's own identity, and encrypts this using Hal's public key, and sends this string of stuff to Hal on redtooth 7.

Hal decrypts this using his private key. ONLY someone with the private key can do this in reasonable time. Hal then looks up the public key of matching the cop's computer's identity, re-encrypts the token, along with the latest date/time stamp, and some random noise, and sends this back.

The cop's computer decrypts this data using its private key. The tokens match, proving that Hal was able to decrypt the packet, and hence actually has Hal's private key.

More about public/private keys https://en.wikipedia.org/wiki/Public-key_cryptography

$\endgroup$
0
$\begingroup$

Not feasible.

All methods proposed are fundamentally flawed.

You can, up to a point, certify hardware. Some hardware include some "write-once" memories very difficult to "spoof".

You can also "homologate" procedural software using signed signatures and be reasonably sure what's actually running in the hardware hasn't been tampered with.

Unfortunately nothing of this is applicable to a reasonably complex A.I.

Actual knowledge in a modern A.I. resides in data, this may be the weights defining a Simulated Neural Network or some Content Database or something yet different, but, in all cases, dynamic in nature and thus defying some simple(?) signing scheme.

Normal I.D. cards rely on fact "hardware" is rigidly connected with "software" and there's no way, at least in the foreseeable future, to "reboot" some Natural Neural Network with a different "Operating System".

The fact "Software" is not tightly linked to underlying hardware (and many layers ensure a program doesn't even know on which platform it is running, at least in many modern Programming Languages) is a game-changer.

You can easily "clone" a program and have it run on many different hardware platforms; Should all of them have the same "I.D." card?

Even if they started "identical" thy are bound to have different "experiences" and thus diverge, possibly in radical ways; At which point your "ID." should be invalidated?

No. The only potentially viable approach is Python's Duck Typing. Prepare specific tests (they can be administered quite fast due to electronic speed) to test if a certain program can (still) perform a certain task.

In the specific case this is equivalent to have the "policeman" connect some testing device to A.I. and have it pass a full fledged driving license exam. No other test would provide any reasonable answer.

Note these problems are similar to what happens with on-line identification which is much less reliable then many people think (as the many "identity theft" cases testify) and essentially works well (when it works) by providing feedback to user about all transactions, so that they can be traced and stopped, not by preventing abuse.

$\endgroup$
-1
$\begingroup$

It's not really a problem.

An AI will (presently) never attain a valid driver's license because the law simply doesn't allow for it. That means that every car driving mostly (or even fully) autonomically must have some sort of exception which, due to the associated cost, is valid for the car model (not an individual car), and only valid with some other contraints (such as a human driver with a valid license who is finally to be blamed if the AI drives into a truck). A prototype, likewise, would have an exception, but could never get a proper license.

Unluckily, at some point in the hopefully very distant future, there will be a desire for AIs to have proper driver licenses. When that happens, it is up to the legislative to come up with a solution.

The identity problem could be solved with readily available cryptographic algorithms. A simple digital signature would do just fine. This could come from the drivers license office if you like, but the maunfacturer could as well do it as long as their signing key is either registered in a central database or properly signed certificates in a state-controlled hierarchy are supplied (much like in every web browser nowadays, except now it's more a wild west hierarchy).

Or, seeing how it's well known as a trope that AIs are not allowed to lie, you could simply ask the AI for its identity and trust it.

Realistically, you know what will happen. Legislative always goes the most stupid, most sure-to-fail, and most needlessly expensive way.
Which means cars with AIs entitled a driver's license will either simply have magenta-colored license plates (not to be confused with fuchsia-colored plates which require a human driver with a license), or you will have a sticker in the front window, and you will not be allowed to change plates or remove the sticker. The sticker will cost you approximately 500 dollars per year, and you will need an additional sticker (valid for 8 months) if your AI is qualified to drive through a tunnel.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .