The closest analogues to what you describe are the Ballista (torsion catapults) of ancient antiquity. While not using pulleys and cams the way modern compound bows do, they were amazingly intricate mechanisms which could shoot javelins or large rocks hundreds of metres, and certainly not to be trifled with.
Typical ancient torsion powered catapult
Indeed, in Roman times, the technology had become so standardized that the Legions had a man portable device known as a "Scorpion", which served a similar purpose to crew served weapons in a modern army.
Reproduction of a Roman Scorpion
Looking at this, you may wonder why Scorpions and Ballista disappeared before the end of the classical period, and the answer is brutally simple: cost. The materials used to make Ballista and Scorpions would have to be of high quality to withstand the stresses of the torsion springs, but more importantly, a highly trained Ingenium was needed to do the calculations to ensure the torsion springs were tensioned correctly and all the parts were properly assembled. As the resources to maintain this pool of skilled workers declined, the ability to build and use Ballista and Scorpions faded as well.
This leads to why compound bows were not invented in Antiquity, the Middle Ages or in fact until the 1960's. Bows were a very highly developed technology already, with literally tens of thousands of years of development behind them, and specialized bowyers who knew the craft of making them. The principles of mechanical advantage were also well known from ancient times, but the sorts of people who used pulleys (sailors, construction workers) were generally not involved in combat. Interestingly, the two were combined in the late Middle Ages and early Renaissance with the introduction of "spanning" devices to draw heavy steel bows.
Crossbow with windlass spanning mechanism
And this explains exactly why compound bows were not invented until the 1960's. A steel bow with a windlass spanner could have a draw weight of up to 1200 lbs, providing more than enough energy to propel a quarrel capable of killing or injuring an armoured man. Both the bow and the windlass were relatively inexpensive, and were simply extensions of existing arts. No highly trained engineers would be needed to set up the bow, or make the adjustments needed to maximize the energy delivered, and both the bow and spanner would be relatively simple to repair in the field by currently trained bowyers and engineers.
So the issue isn't that people were not "smart" enough to invent compound bows, there simply was no need for compound bows in that period, since existing equipment could be made, maintained and used by people with existing skillsets, and was quite capable of doing the jobs demanded of them. Extra costs will always be ruthlessly pared away, and the cheapest, most capable pieces of equipment will generally be what you will be using.
