Short answer:
Using Hexadecimal is more efficient than decimal for a telephone system that uses digital logic to connect two telephones together based on a phone number.
This is a simplified example, but lets say a decimal digit in my phone number selects a choice for some relays in my telephone system, and those choices are being controlled by digital logic, then I need 4 bits to represent that choice. But digits 0-9 only use 10 of the 16 possible choices for those bits. I have wasted hardware because those 4 bits could be used to turn on one of 16 relays, but I am only using them to control 10 relays. My system becomes larger and more costly to support the same number of customers compared to using hexadecimal phone numbers.
Long answer:
One might argue that the development of telephone systems was a complex task at the time it was first undertaken. In your world the engineers needed to squeeze every last drop of efficiency out of the system to make things fit within the constraints of size, weight, power, and cost. In that case hexadecimal was chosen for the sake of efficiency.
Now let's go into why it's more efficient. Early telephone systems were completely analog, with switchboard operators manually connecting people by plugging wires into different holes.
The next phase in the evolution of the telephone system is replacing the human telephone switchboard operator with an automatic system that connects the different telephone circuits. That would likely involve relays. Relays are open/closed, on/off. Given that relays are the state-of-the-art technology of the time, and the phone companies are already using them, any other logic that decodes the phone numbers to control the signal relays would likely be composed of more relays. The relays are being used to make digital electronics.
For electrical engineers who are designing digital computer equipment, number systems that have a power-of-two as the base are typically more efficient. Even more efficient is power-of-two bases where the number of bits used to create that base is also a power of two (1, 2, 4, 8, 16, 32, or 64-bit numbers).
Some examples are below.
- 2^0 bits = base 2 = straight binary.
- 2^1 bits = base 4 (not commonly used anymore)
- 2^2 bits = base 16 (hexadecimal)
- 2^3 bits = base 256 (the common 8-bit byte)
- 2^4 bits = 16-bit numbers (common register size for 16-bit processors)
- 2^5 bits = 32-bit numbers (common register size for 32-bit processors)
- 2^6 bits = 64-bit numbers (common register size for modern 64-bit processors)
Of the above choices, straight binary take a lot of digits to write numbers (about 10 bits for every 3 decimal digits) and is therefore not very compact. Creating a symbol for each of the choices in an 8, 16, 32, and 64 bit number would result in way too many symbols for humans to memorize.
That just leaves 2-bit and 4-bit numbers. The 4-bit numbers only have 16 symbols to remember, and one can express numbers fairly compactly (about 5 hex digits for every 6 decimal digits). That of course explains the ubiquitous use of hexadecimal in computer programming and electrical engineering.