Pay a visit to any semiconductor manufacturer and the answer will slap you right in the face: cleanliness. There's a darn good reason why all the fab workers wear moon suits, and they won't let you anywhere near the shop floor without one. So what if you contrive a situation whereby creating a sufficiently clean environment is impossible? Or the required purity of raw materials is unattainable?
It turns out that these sorts of advances were made circa 1970 to enable building ICs using the NMOS transistor, which has significantly better performance than a similarly sized PMOS transistor, and consumes less power than BJTs. If you are limited to PMOS and BJTs, you can still build reasonably complex ICs, but you will be hamstrung to a roughly 1980s level of technology at best.
In particular, you cannot build CMOS chips without NMOS transistors; though you could build roughly analogous structures using BJTs, they would still be larger and consume considerably more power. But you could still build depletion-load PMOS chips using the same ion-implantation technology as enabled depletion-load NMOS, and that would be important in trying to build 1980s designs.
This would also exclude the larger and more ambitious chip designs of the 1980s from consideration, or at least make them very much more expensive. The 6502 has about 5000 transistors, which should be quite manageable, but the 68000 has as many as 40000, and was thus extremely expensive to produce when first introduced in the late 1970s (with the benefit of NMOS transistors). For comparison the much more capable ARM2 only used 30000, and the 8086 used about 20000.
The 6809, 6309 and 65816 might be representative of more achievable designs. The 6809 was a basic 16-bit CPU using only 9000 transistors. The 6309 extended the 6809 in some logical and useful ways, and probably stayed well under 20000 transistors. Meanwhile the 65816 was designed as an upgrade of the 6502 family with the minimum of additional hardware, and may have stayed under 10000 transistors, though with some significant missing features relative to the 6309.
Larger computers could still be built using SSI and MSI technology, as many of the famous DEC minicomputers were. The principal limitation to these was the sheer cost of installing large memories; this could be worked around to some extent by using disk and tape storage, at a significant performance cost. Their relatively high power consumption and frequent need for repairs would also limit the number of people or organisations which would bother to install one, if even a basic 6502-based microcomputer was widely available.