You have provided with a preformed structure for your world:
You can't easily transmit ciphered information (keys, passwords, etc.) and you can't hide anything but with physical vaults.
Instead of arguing whether or not this is practically possible (with current technology), let us concentrate on your questions.
What would this society look like?
Almost the same as it looks like, now. Only two-key (public and private) type encryption systems would not be possible. The passwords on emails and user accounts would still be as effective as they are now.
Simply, because hacking a password is an extremely long and boring process known as "brute forcing", where the attacker tries every possible word combination possible for a given length of characters. This is the only type of hacking possible when you don't have a direct attack on the information possible (keylogging, server database hack etc).
Yes, encrypted connections between server and client side computers might not be possible in such a world and hence banks would have limited services regarding online transactions and money transfer.
Militaries all over the world would have a very high reliance on codebooks than on digital encryption systems.
How could those people still transmit secured messages? Is it possible?
Yes it would be possible. People would send encrypted messages to each other without sending the key to decrypt those messages. This is how the system would work:
Person A sends person B an encrypted message
Person B applies another layer or encryption on the message and sends it back to person A.
Person A removes his layer of encryption and sends the message to person B.
Person B removes his layer of encryption and reads the message.
At no point in this protocol does a transmission of keys occur. Yes, this system might be slightly slower in working than the protocols we have in place these days, but it would still work flawlessly for digital communications.