What you're describing, if I'm understanding correctly, is either a Night-Watchman State, or a something of an Anarcho-Capitalist society. I have heard arguments for this sort of government, but my concern is that it would create a highly inequitable and unstable society. Can it be done? The answer of course is yes, but it probably wouldn't last long.
To try and contextualise this, understanding the history of capitalism is essential. Modern conservatives in Britain view pre-Socialist Britain as something of a paradise lost, a view encouraged by Margret Thatcher. But it's a gross and erroneous simplification. I will try and keep this brief (and fail).
After the Second World War Britain held a general election in 1945, which was won by the then socialist Labour party. They adopted sweeping reforms, such as universal healthcare, unemployment welfare, free education, slum clearance, nationalisation of industry, etc.
These reforms came about as a result of popular frustration with the status quo, and the realisation that nationalised schemes could work. After all, if "war communism" could defeat Nazi Germany, surely other collectivist systems could work during peace time?
The Liberal Reforms of 1906-1914 were insufficient. They had been passed to tackle social problems endemic within the classical liberal society of the 1800s. New state-led policies were implemented; like a state pension, national insurance (healthcare), and job centres. Before these reforms the elderly, sick, and unemployed had to rely on family and charity, which was often just not enough.
Poverty was endemic in Britain, even by the start of the war in the late 1930s. There were attempts to help, like Victorian Workhouses, but these were often little better than Debtor Prisons. The gross inequality also led to security concerns, for example Anarchist Terrorism in London, and sympathy for communism.
A frequent Anarcho-Capitalist argument is that a stateless system will balance upon competition between insurance companies, working in the interest of the consumer. Insurers contract security firms to enforce law, which will insure peace and stability. But this seems something more like a society run by fancy mafia clans. Even in the contemporary world where the state is a strong institution, corruption, bribery, and dirty tricks are frighteningly common when it comes to big business.
There are historical examples contrary to the notion that corporations will work for the consumer's interest, like the Pheobus Cartel. This was a conspiracy of lightbulb manufacturers between 1924 and 1939, with the largest manufacturers creating a monopoly, and agreeing to both control prices, and creating lightbulbs which would break more often to increase sales.
Without deliberate efforts to mediate between the interests of workers, government, and corporations, inequality will increase and this will lead to poverty, anger, social unrest, and perhaps even revolution. Don't forget, that's what happened as a result of the Great Depression in 1929. Germany in particular was vulnerable at this time, owing to its reliance on American investment.
When the depression happened this dried up, and every single German bank failed. With this the entire system, which had seemed so secure in the roaring twenties, was in peril as the nation was plunged into poverty. There were street battles between communists and fascists, as well as communist and fascist revolutionaries. And we know which ones won - on the premise that liberal capitalist democracy is weak.
NASA has even done a research paper on the collapse of civilisations, putting inequality high on the list of causes. Concerningly, Thomas Piketty's magnum opus; Capital in the Twenty First Century, finds that after an analysis of capitalism from the birth of the industrialisation to present, there's one obvious rule. Private wealth grows faster than the economy, therefor inequality is natural to capitalism. Without efforts to mitigate this by organisations which have an interest in helping the poor, we are led back to the Great Depression and NASA's paper.
So given these things, it seems likely that a "pure" capitalist society would come into being, become very unequal, and then be overthrown by revolutionaries of some description. It could also tear itself apart as rival insurance and security firms compete for turf, and descend into feudal or criminal mentalities. Mexico's drug war for example.
How could this happen? Look at western politics since the end of the Second World War. We're already heading in a classical liberal direction. Lower taxes, lower taxes, lower taxes. Sounds great. But. There's a relevant phrase here: With taxes I buy civilisation. I suspect the problem is a very short collective memory. None of us remember what it was like in 1800 or 1900 or 1928, and so we don't think that it could get like that again. Because we are generally dangerously optimistic.
