We often think of things like nations as being neat entities with clear cut boundaries and rules, but the reality is that the state exists only where it can enforce itself. Some places are controlled by defacto states under a dejure national blanket, be they criminal, terrorist, or religious. I'm going to discuss the issues and potential solutions you may encounter on your quest to solve the petty crime problem, based on historical examples.
It has been said that in Italy the Mafia and state are separate, but in Russia the mafia is the state. Mafia clans are erratic, and will declare war or peace amongst themselves, or against the state; whatever suits their interests at the time. Italy's Second Mafia War resulted in thousands of murders. That is something of a contrast with the years immediately after the Second World War, when the CIA encouraged an anti-communist pact between Mafia, Catholic Church, and Italian state. Italian-American mobsters in US jails were released, given a one way ticket home, and told they could do as they liked so long as they kept communists out of government with the country's first election looming.
Modern Russia has to be understood through the collapse of the USSR. The KGB, rebranded FSB, became an extremely influential organisation as the rest of the country fought amongst itself for control of former state assets. Little surprise then that Putin, as the former head of the FSB, became president. Russia's culture of corruption is perpetuated from the top down, and money filters upwards through a "power vertical". Police extort money so long as some of it goes upwards. Organised crime exists because of implicit government support. There isn't really much of a distinction between the state, its organs, and any serious organised crime. Certain things are tolerated so long as they pay tribute to and follow orders from the top.
Northern Ireland is another interesting example. During The Troubles many parts of the country were controlled by terrorist organisations from either side of the conflict. In communities dominated by the PIRA; where nobody would go to the dejure British policing and legal authorities, they acted as judge jury and executioner. Conducting many executions allegedly to punish British spies, and conducted many kneecappings allegedly to punish petty criminals. One PIRA operation in 1992 called "Night of the Long Knives" saw them chasing a rival Republican terror group out of west Belfast; allegedly because the group's feuding and drug dealing was becoming an embarrassment to their nationalist cause. Of course, presently Sinn Fein have been in power sharing local government for years; and it's important to remember that PIRA/Sinn Fein are two sides to the same coin. Organised crime in Sinn Fein areas happens under PIRA sanction; otherwise the issue would be resolved by PIRA men as they did in the past.
So there are a few ways your state can handle the crime problem: tightly coupled, loosely coupled, or decoupled.
Tight coupling: like PIRA/Sinn Fein, means by design both sides of the system are stitched together. Of course, officially your politicians were never active members of X. But we all know that's not true. Indeed, without having been an active member of X it would be impossible to achieve political influence. Their reputation came from their personal involvement in "the struggle". The government has intimate knowledge and control of exactly what is going on at every level of society because it exists as a single institution throughout. Individuals can be disciplined because the organisation has a large pool of individuals under its direct command who can replace their peers.
Loose coupling: like Putin's Mafia State. This is a system with clearly defined boundaries between entities. Not only can association be denied, but the left hand doesn't necessarily know what the right does, or indeed who it is. This is unlike the former solution where status is common knowledge fundamental to the system's stability. The power vertical doesn't care exactly what any of its organs do, so long as they do the job and play by the rules. Those at the top, who likely belong to a state intelligence organisation, are aware of everything and hoard knowledge. The downside of this indirect control is that excesses can occur, and they are harder to deal with because there are fewer links in the chain; which makes those links less disposable.
Decoupling: like post-war Italy. Tasks are achieved by completely separate organisations whose relationships are based on trust instead of control, with very little operational knowledge of each other. This is unlike the previous two examples, in which a central organisations has command and control capabilities over most aspects of society. The downside is that when circumstances change violence between former allies is possible, and over the long term this is inevitable. This system however benefits from the highest degree of plausible deniability. Of course the president has nothing to do with X. But he did order his subordinate to send commands to them at a secret meeting.