Planet A has alien species A. A, before becoming spaceborn stumbles upon several factors and pieces of technology from another (long dead) alien species B.
A attempts to reverse engineer B's technology but finds it so fundamentally alien that all efforts to discover any meaningful advances from it fail spectacularly. They can, however, use the technology directly, and use the factories to repair and reproduce the (unchanging) technology. It's also reasonable that they could produce their own look-alike but more primitive technology given enough time.
The government in power over species A doesn't want this weakness to be known to its citizens, but does wish to exploit species B's technology.
Later, as they take to the stars, they run into other alien species which they eventually form a trade network with. They only assume that they will have to face spies (or other intelligence gathering techniques) attempting to steal their technology.
How would the government of species A manage to hide the fact that they don't actually know how their technology works from both their own citizens, and later the galactic community?
I'd rather not resort to religion or superstition, and nothing like magic or deities exist in this setting, but I'm perfectly willing to have the government heavily organize their culture over the decades/centuries.
The end idea is to have the alien society as a militarily weak power that uses threat of incredible retaliation to ward off warlords and tyrants. They pull this off by regularly showcasing extremely advanced technology, and nobody able to crack how it works. Unfortunately, they can't crack it either, and desperately need to keep this a secret, as technology you can't easily repurpose to intended ends has limited uses.
I haven't decided at all what this technology would be but it should be readily and constantly apparent to anyone who trades with them or visits their planet, quite possibly a host of different bits of technology, if it be chemical engineering, computational devices, hovercraft, or whatever else.