Who we are is worse than who we wish to be.
Our relationship with machines is only representative of our id, but our dogmas that we verbalize to other humans when our reputations are on the line better characterize our egos.
Spyware on a person's computer is likely to dredge to up all of that person's hidden frustration, kinks, and buried prejudices. But people don't just act on what we think and feel, we act on what we aught to think and feel too. And more importantly, we base our rules and laws on how people aught to think and feel.
Everyone knows speeding is wrong, and we all want everyone else held accountable for not speeding... But everyone complains about speeding tickets and speed limits to. Listening only to people's online rants, an AI could easily assume everyone hates speed limits and they should be gotten rid of, but just because the law frustrates us does not mean we think we should get rid of it.
The representative process better represents our ego's desire to impose not just our wants, but also our beliefs on the world around us. Representatives, or more importantly, the act of getting together as humans to discuss issues forces people to act from our Egos. This causes us to suppress a lot of those hidden prejudices that are more likely to come through in the anonymousness of the internet than they would when talking an issue out with other people face-to-face.
AI does not understand how hard life is.
There are many laws that sound good on paper but are intractable in practice, or the inverse, laws that sound terrible on paper, but in practice represent the best possible compromise between negative outcomes.
For example, Section 230 is one of the least popular laws in the American legal code. Most people hate it for one reason or another, on the left people hate that big tech companies are immune from liability and can host false, misleading, and hateful content free of consequence, and on the right people hate that it gives tech companies unlimited power to censor opinions and violate people's constitutional rights. If an AI were just going off of public opinion, it would nix this law in a heartbeat.
But this law exists for a very fundamentally human reason: If you modify this law in either direction, it would effectively kill the internet. If you take away a company's right to take down, block, or bury content, then it easily becomes a platform for scammers, hackers, terrorist groups, etc., and if you make these companies liable for what they show, then the risk of hosting a search engine would far outweigh the profit margins. Google, Facebook, Twitter, Stack Exchange, etc. would simply be better off going out of business and liquidating thier assets than trying to function in such a world.
An AI powered by the personal opinions of the masses would not understand or care about the human impact of changing such a law, but a well informed human representative can easily see the merit in it. So, by allowing humans to decide when an unpopular law is also a needed law, you ultimately lead to a less dystopian legal system where the law represents not just what people want, but temper it with human understanding of when getting what we want is just plain too hard to actually live with.
Balancing AI and Representation
The AI knows everyone's personal thoughts and feelings more than it understands how people treat each other face-to-face on a daily basis, or how all the parts of our society impact one another. So, while it knows its not the best candidate to make humane choices for us, it can certainly do a better job than we can at picking out what a statistically representative group of humans looks like.
So, while the AI's lack of "real world" data may make it a poor choice for what the law needs to be like, it could do a much better job than humans at electing the representative body. By aggregating human personality and background traits into cohorts, and picking people from those cohorts members that have important leadership qualities like verbal skills, legal understanding, altruism, etc. The AI could elect the best humans for the job, and then leave the actual job of legislation to these individuals.