Tl/Dr; Yes, I think it is possible. However, it is not easy. In fact, its hard to even pin down the exact requirements on the question. The real key is that you have to model the universe with yourself in it, which encourages solutions that are self-similar, fractaline in nature.
Given that questions like these are the subject of the rest of the Foundation series, I'm going to consciously deviate and consider another character which predicted the future: the Paradash Emperor Duke of House Atreides, Paul Atreides, aka Muad'Dib. For those unfamiliar with the name, he is a character in Frank Herbert's Dune. The theories one may apply to Paul certainly apply to Hari Seldon. By switching to a different universe, I can talk about common issues they may face using details and word choices which hopefully will not lead to spoilers.
This answer is long. Perhaps too long, but I find it actually ties into real life applications well enough to put it on paper. We humans do too many things that look like miniature versions of psychohistory to leave the concept to fiction. The first part discusses what to do with the chaotic systems that wreck typical predictive algorithms. The second part explores why working with other humans could be a very wise path. The final part gets to the real crux of the issue, of why psychohistory could work, or why it couldn't.
The first step to a question like this is going to be defining some basic assumptions. The first thing needed to really make any headway on fantastical theories like Psychohistory is compatabalism. Modeling the freewill of a human being is tough. In fact, its a major sore point in Western philosophy. Compatabalism claims that the freewill we have is compatable with a physical world: we do not observe anything which requires a metaphysical world above. In Dune, Frank Herbert never relies on anything supernatural which could not be explained using physics. The great mythos of Shai Hulud is explained as the result of centuries of gently prodding by the Bene Gesserit and efforts in planetary ecology. The place the Bene Gesserit cannot go is highly spiritual in its description, but the explanation for what it is and why they cannot go there is defined within the bounds of physics.
As a result, it is possible to simplify the problem at hand by transforming the world we live in to a world which is identical in every way, except every living being with freewill is replaced with a p-zombie, a philosophical construct which has no freewill but acts exactly the same as though it had it. If one can find a solution to the problem in this world, it is trivial to show that it must work in this world, but this is only true if one starts from the assumption of compatabalism. Without that assumption, the possibility of things like miracles really limit the ability to predict the future.
The second assumption I want to make is a limitation of scope. Saying "is is plausible that psychohistory works" is reasonable, but to come to a consensus on whether or not it is actually plausible, I think it's helpful to pin it to more strict wordings. There are two requirements that I believe make sense here. The first is that the the theory must yield admissible results. "Admissible" is a statistical decision theory term which basically states that there is no rule which is always better than it. This is a low but essential bar. Playing the lottery is an admissible decision because there is a possibility that you will end up richer than any other decision you could have made. It is a good rule because it handles the case where you simply don't have the cards in your hand to save the universe. You don't have to "win" the game, you just have to play a hand which isn't provably worse in all cases than a different play.
The final requirement I add is a minimum threshold for utility. You want to accomplish more than just doing nothing. In our real life, this can be hard. There's a lot of amazing stuff out there that doesn't require our hand at all. However, in both Paul's world and Hari's world, that isn't the case:
- In Paul's world, the genetic essence that makes us "human" is becoming too dilute to sustain itself. Failure to do anything could result in the end of all humanity.
- In Hari's world, the dark ages are coming, 10^30 years of darkness are at risk. Needless to say, the future is dark indeed.
In both cases, we see that there's a lot of room for improvement. The darker the world is without intervention, the easier it is to argue the value of such intervention. It will also help because you are going to have to make some very tough calls along the way. If the world could turn out just fine on its own, it may be very hard to justify acting in a way which can harm the structure you're trying to create.
Now the key to this puzzle is trying to figure out what sort of thing we're trying to construct. We know it needs to be able to do predictive modeling on a large scale, using small scale resources. From an information theoretic standpoint, this likely means the need to aggregate large amounts of state information from the world around us into small scale information. We want to be able to play with numbers like "GDP of the United States," not "a list of the GPS position of every dollar bill, by serial number."
Obviously, if we had such an aggregation system in place, it would be easy. However, that's handwaving a mighty thing. Consider what anyone would give for a tool that could predict the things that matter in their life. We can't just handwave away the construction of this system, it has to at least be plausible to create.
The system also has to deal with chaos. It is currently believed that the world contains chaotic systems, such as weather. They are unpredictable, by definition. The obvious solution is to try to ensure no chaotic system's behavior matters in the great scheme of things, but we run into a self-referential problem. We have to treat ourselves as a p-zombie, if we want to be consistent, and thus our unpredictability and chaotic behavior would need to be squelched like all others. A smooth laminar universe awaits!
'Tis a horrible idea no? Okay, lets work on a far harder answer: embracing the chaos. How can we accept chaotic systems into our tool and let them help us? The secret lies in these metronomes.
We can entangle ourselves with these chaotic systems to predict and affect their state. This is similar to entangling photons so that their states are the same at great distances, except it will be done macroscopically, and has to be done without the help of Quantum Magic. The goal is to create a chaotic system within your tool whose state is coupled to the chaotic system outside of you by continuous interaction. Over time, you can build up a feedback loop such that the behavior of the two systems is linked. If one of them is perturbed, it rapidly transfers the information to the other, keeping the two in sync. Its hard to do, but fundamentally very similar to what one has to do to balance a broomstick on one's hand.
So now we have a few useful tools:
- We can synchronize with chaotic systems, allowing us to predict information about their state.
- That synchronization also allows us to know the lowest energy way to affect the state of the chaotic system outsides.
The second half is key. Efficiency of energy utilization is going to be a big deal. The Chinese martial art Tai Chi has a saying, "Use four ounces to move four tons." We're going to use perhaps 4kg to move 10^52kg! Sheesh! We're going to have to be super efficient!
Of course, there's still the question of the systems which are not chaotic. They are well described by classical mechanics, so we can actually measure their state directly rather than having to entangle with them. However, there's a question of storage space. There would be far more data than you could possibly store in your head, and its clear any one bit of it could be useful. We're going to need more storage. Fortunately, we have somewhere to store it -- all of the chaotic systems we entangled have plenty of state data which we aren't 100% correlated to. Unfortunately, chaotic systems are a terrible place to store information. We're going to have to work on that next. We're going to have to enlist chaotic systems to store and process data for us. That's right, enlist chaos. If you try to skirt around the issue of chaos by damping it out, psychohistory is impossible.
You will likely find some chaotic systems more helpful than others. For example, your fellow compatriots, or at least their p-zombie representations, can help. They are very chaotic systems that have arguably been bred for observation and data processing. We've got a kilogram of squishy wrinkly material in our head dedicated to the task! This shows up in Dune with Paul's choice of companions. The Duke, and later Emperor, is always surrounded by people who are observing the world, processing information, providing it to him, and acting on his orders (which includes guarding his life).
It was intuitive to see that your best friend is going to be of more help for psychohistory than a gust of wind, but why? If we understand that answer, we may be able to better choose which people/p-zombies to bring close to us. I think part of the intuitive answer is trust. You need to be able to trust these people to act as an extension of yourself. You need to be able to trust that they are holding onto the key information about some system, and are monitoring it for any changes that you might need. You need to be able to trust them to act on their own. What sort of system might we be able to trust in this way?
One of the key facets of trust is that it is built through repetition. You prove your trustworthyness by being predictable and reliable. Of course, the ground beneath your feet is predictable and reliable, which demonstrates that the trust we need also needs to be flexible, or else you're treating your best friend like the dirt beneath your feet.
The simplest structure which is trustworthy is one which polices itself. If you can trust someone's "inner self" to police their "outer self" and keep it in check, maybe even report to you when the outer self is misbehaving, then you can trust the person as a whole. Of course, this is a chicken and egg problem. You have to be able to trust the "inner self", which could involve trusting an "inner inner self" and so forth. In an ideal world, such as one where psychohistory might come about, this process could continue as deep as one needs. In our world, we know eventually there's going to be something unpredictable at the core of this structure. We'll address this in a moment, but there's an important and convenient detail here. We mentioned earlier that our plan needs to include the effects of our own actions. This means a sub-plan of the plan needs to account for whatever we choose to do. Thus means a sub-sub-plan of the sub-plan needs to account for how we plan for our choices, and so forth. We need the same structure in ourselves as we need in those around us! This is very important, because it permits us to be part of someone else's psychohistory while we strive to be part of our own!
We see this in Dune. Paul's closest compatriots, such as Gurney Hallack, Stitgar, and Paul's own mother and father, are all striving to shape the world to their goals, and all have this deep self-within-a-self structure. Each of them encourage Paul to develop his own structure deeper, both helping them with their goals and helping Paul achieve more control over his own future. This can be contrasted with the Baron Harkonen. Under the Baron, depth is only encouraged as long as the Baron maintains control. He takes care to make sure none of his servants gain more control over his life than he has over theirs.
Now with all of this, we can start to put the pieces together. We can see how we can entangle ourselves with chaotic structures, including our p-zombie friends. We can see how to use these friends to keep track of the huge volume of valuable information we need for our psychohistory. And we can see how we can work together, even if there isn't a strict hierarchy of who is in charge. The lack of a strict hierarchy brings up an interesting question: whose psychohistory is it anyway? Did Hari Seldon really create it out of the blue, with no external muse guiding his way? In Dune, Paul Atraedes clearly admits he's following some greater scheme that he does not fully comprehend up until the moment where he does, and crowns himself Emperor of the Galaxy in the aftermath.
So we clearly have to consider two patterns. One is that there were already forces at play trying to solve the dark ages problem of Foundation, or forces at play leading towards the unraveling of the dangerous web the Bene Gesserit wove. In such a case, psychohistory or one's status as the Kwisatz Haderach is really just the culmination of a much larger effort. It may seem magical, because nobody can see the undercurrents. This would suggest that it is no accident that Hari took the secrets to his grave.
The other pattern is that your Hari or Paul is actually the first ripple forming in an avalanche that follows. In this case, Hari or Paul may have undergone tremendous inner searching, and found everything revolved around one small nugget... the part of themselves they could not see into. They brought it to the surface, and it happened to be more effective than any other approach anyone else had. If someone else had an effective approach, Hari may have believed psychohistory works, when it in fact does not. In Dune, a similar thing happens when Paul's son actually disrupts Paul's vision of the future by doing something Paul never thought someone would do.
In this case of a small influential nugget, it might expand with the flow of time like a smoke ring, until it is entangled with everything of importance. Then, it may turn inward and find a Hari or a Paul to be its harbinger. However, what kind of idea can behave this way? The more divisive the idea, the more rapidly it devolves into Baron Harkonen's style of thinking. The more unifying the idea, the more it can be upheld by those like Paul's gang of highly conscious individuals. In fact, the most unifying ideas would be those which so strongly resemble the status quo that nobody realizes anything changed at all!
The most powerful ideas may be no more than just a whisper.
