Only the people go back. And incomplete knowledge is especially useless without the tools to exploit it.
Or, in other words: if you woke up tomorrow, 1000 years in the past, with nothing but your sleepwear, would you be able to build a computer? How well could you pass all your modern knowledge on to your children and grandchildren?
Your time-travellers scattered throughout history. Some arrived yesterday, some arrived last week, some arrived millennia ago. But very few of them know how to properly prove their knowledge from first principles, and even fewer know how to bootstrap the technology from near-scratch. For the most part they are, at best, regarded as "not quite all there". Further down the scale, words like "crackpot" start to be thrown around.
Ironically, those who went back further are better off from that perspective — modern science requires empirical evidence, or repeatable experiments. The Ancient Greeks just needed you to be suitably skilled at oration and debate.
In an era where signs of futuristic medical care are even more impressive, your time travellers may even be able to rally tribes together, and join forces to build great and advanced civilisation. But, evidently, something went wrong: Atlantis, Mu, Dvārakā, and Shangri-La are now only spoken of in myths and legends.
The end result? Knowledge is lost, corrupted, or decays. Society advances… and winds up exactly back where they started. They don't advance further, because they still have to spend so much time recovering what was lost.
So turns the wheel of time.