Hypercomputation
According to Wikipedia Hypercomputation is defined to be the following:
Hypercomputation or super-Turing computation refers to models of computation that can provide outputs that are not Turing computable. For example, a machine that could solve the halting problem would be a hypercomputer; so too would one that can correctly evaluate every statement in Peano arithmetic.
The Church–Turing thesis states that any "effectively computable" function that can be computed by a mathematician with a pen and paper using a finite set of simple algorithms, can be computed by a Turing machine. Hypercomputers compute functions that a Turing machine cannot and which are, hence, not effectively computable in the Church–Turing sense.
Technically the output of a random Turing machine is uncomputable; however, most hypercomputing literature focuses instead on the computation of useful, rather than random, uncomputable functions.
What this means is that Hypercomputation can do things computers cannot do. Not in terms of scope limitations such as the ability to access things on a network but rather what can and cannot be fundamentally solved as a mathematical problem.
Consider this. Can a computer store the square root of 2 and operate on it? Well maybe because it could store the coefficients of the polynomial whose solution is that square root and then index the solutions to that polynomial. Alright, so we can the represent so called algebraic numbers (at least I believe so). What about all real numbers? Euler's constant and pi are likely candidates for being unrepresentable in any meaningful sense using binary. We can approximate but we cannot have perfect representations. We could have pi be a special symbol as well as e and just increase the symbolic set. Still not good enough. That's the primary thing that hops to mind to me at least. The ability to digitally compute any real number with perfect precision.
This would be a reason for such a society to never discover binary computers being useful. At some point we switched from analog to binary because of electrical needs and signal stuff. I honestly do not know the details. We modeled the modern notion of processor and other things loosely off of the notion of a Turing Machine which was ultimately the form way of discussing computability which was kind of a multi faceted convergence of sorts. There was the idea of something being human computable and then theoretically computable. The rough abstract definition used for many years ended up converging with that of the notion of the Turing Machine. There was also the set theory concept of something or other (I don't recall the name) that ended up also converging to defining the same exact same concept of "computable". All of these converging basically meant it was said and done. That is what we as a society (or even as the human race for that matter) were able to come up with as a notion of what is and is not programmable. However, that is the convergence of possibly over 3000 years of mathematical development possibly beginning as far back in concept as Euclid when he formalized the most basic concepts of theorems and axioms. Sure math existed but it was just a tool. Nobody had a formal notion of it. Things are just obvious and known. If Hypercomputation is possible for humans to do (rather than it just being a thing limited to machines) then all it would take is one genius in the entire history of math to crack that. I'd say it is a reasonable thing for an alternate history.