What you're describing, if I'm understanding correctly, is either a Night-Watchman State, or a something of an Anarcho-Capitalist society. I have heard arguments for this sort of government, but my concern is that it would create a highly inequitable and unstable society. Can it be done? The answer of course is yes, but it probably wouldn't last long.
To try and contextualise this, understanding the history of capitalism is essential. Modern conservatives in Britain view pre-Socialist Britain as something of a paradise lost, a view encouraged by former British Prime Minister Margret Thatcher. But it's a gross and erroneous simplification. I will try and keep this brief (and fail).
Edit: I had written this answer a few years ago, and since then have had some more ideas. The most relevant one is that a small state has minimal power to enforce law. In this case, corruption cannot be moderated. The Mafia are an important case study.
The small state is not a hypothetical future without historical precedent. In 1861 Sicily was annexed as part of the unification of Italy. This led to land reforms, resulting in the privatisation of public and church land, increasing the number of land owners tenfold. The transition from feudalism to capitalism was problematic, as the new Sicilian state did not have the resources to enforce rule of law. Having lost their feudal commons, impoverished peasants often resorted to theft. Coupled with rapid economic development, property owners hired mercenaries to guard property and settle disputes. These mercenaries became dynastic entities (the first Mafia clans). As far as I know, the only time the Mafia clans in Sicily were subdued, and life returned to a semblance of legal normality, was, rather paradoxically, due to Fascist Italy.
When the Soviet Union imploded the same thing happened: the power vacuum which resulted from the failure of public institutions was filled by gangs and oligarchs. The minimisation of state authority and privatisation of state security creates a new aristocracy of criminal dynasties. A relevant term is "state capture". Without a strong and independent state, and thus rule of law, there is no ability to enforce contract law and thus no financial freedom. Workers will be exploited by those with money because there is nothing to stop this from happening, the "economy" doesn't care about life or justice or efficiency. All that's left is bullying tactics, which work with neither unions nor government strong enough to act (e.g. Ford's anti-union antics)
Ideas about a small state rely upon the assumption that individuals are rational consumers, who obey logical Game Theory principles about decision making. But the vast majority of people are not 'homo economicus'. Human decision making is, neurologically speaking, emotional. Humans with brain injuries to emotional centres also demonstrate a profound inability to make any decisions about anything (See Antonio Damasio's book 'Descartes' Error').
Historical Context
After the Second World War Britain held a general election in 1945, which was won by the then socialist Labour party. They adopted sweeping reforms, such as universal healthcare, unemployment welfare, free education, slum clearance, nationalisation of industry, etc.
These reforms passed because many people were crushed by poverty, due to a lack of opportunity and thus freedom, coupled with the realisation that nationalised schemes could work. After all, if "war communism" could defeat Nazi Germany, surely other collectivist systems could work during peace time?
English industrialist Seebohm Rowntree published 'Poverty, A Study of Town Life' in 1901. This was one of the first major sociological studies, which surveyed over 40,000 people living in York. The conclusions were simple: urban poverty was not limited to London, and most people struggled financially because of reasons outside of their control (sickness, disability, and wages not matching living costs). The study provoked political debate, and challenged the notion that poverty was just due to laziness, leading to reform a few years later.
The Liberal Reforms of 1906-1914 were insufficient. They had been passed to tackle social problems endemic within the classical liberal society of the 1800s. New state-led policies were implemented: state pension, national insurance, and job centres. Before these reforms the elderly, sick, and unemployed had to rely on family and charity, which was not enough.
Poverty was endemic in Britain, even by the start of the war in the late 1930s. There were attempts to help, like Victorian Workhouses, but these were often little better than Debtor Prisons. The gross inequality also led to security concerns, for example Anarchist Terrorism in London, and sympathy for communism.
A frequent Anarcho-Capitalist argument is that a stateless system will balance upon competition between insurance companies, working in the interest of the consumer. Insurers contract security firms to enforce law, which will insure peace and stability... but this is just Sicilian history writ large.
There are many historical examples contrary to the notion that corporations will work for the consumer's interest, like the Pheobus Cartel. This was a conspiracy of lightbulb manufacturers between 1924 and 1939, creating a monopoly which controlled prices and created lightbulbs which would break more often to increase sales. More contemporary scandals of note include the Volkswagen emissions scandal, and the DuPont PFOs pollution scandal, etc.
Without deliberate efforts to mediate between the interests of workers, government, and corporations, inequality will increase. In turn, this will lead to poverty, social unrest, and revolution. That's what happened as a result of the Great Depression in 1929. Germany in particular was vulnerable at this time, owing to its reliance on American investment. Every single German bank failed. With this the entire system, which had seemed so secure in the roaring twenties, was in peril. Street battles between Communists and Fascists led to revolution; the liberal free market system did not survive.
NASA has even done a research paper on the collapse of civilisations, putting inequality high on the list of causes. Concerningly, Thomas Piketty's magnum opus; Capital in the Twenty First Century, finds that after an analysis of capitalism from the birth of the industrialisation to present, there's one obvious rule. Private wealth grows faster than the economy, therefor inequality is natural to capitalism. Without efforts to mitigate this by organisations which have an interest in helping the poor, we are led back to the Great Depression and NASA's paper.
So given these things, it seems likely that a "pure" capitalist society would come into being, become very unequal, and then be overthrown by revolutionaries of some description. It could also tear itself apart as rival insurance and security firms compete for turf, and descend into feudal or criminal mentalities. Mexico's drug war for example.
Conclusion
This is a great idea for a story, because it allows us to explore popular assumptions about politics, economics, and human behaviour. It would also be exciting given the political instability and conspiracies which would undoubtedly emerge, given all the aforementioned examples... from the emergence of the Sicilian mafia, to the anti-union activity of the Ford corporation (and others), to the rise of Russian oligarchs, etc. Unfortunately, the real world may be moving in this direction anyway, given corruption and state capture.
