More problems than gains
The main problem that makes your concept moot is that the human body can only handle so much recoil before the poor squishy humans that fire the weapons break. And you reach that limit far before you get any practical upshot of using Depleted Uranium (DU) bullets.
There exist sub-caliber bullets (tungsten carbide though, not DU) for firearms today, specifically for rifles. A sub-caliber bullet of high density does not pack more of a punch, but it allows the bullet to fly faster, which makes for a more shallow trajectory and shorter time to target, which makes it easier to hit. I have personally seen the Swedish military soldier handbook for sharp shooter rifles — such as the L96A1 AW — where the difference between using full caliber and sub-caliber ammunition is shown. Unfortunately I do not have it available too show you so you will have to take my word for it that the difference is quite marked.
The same thing goes — partially — for tank ammunition. Yes, it is true that the chemical / metallurgical qualities of DU are advantageous when it comes to defeating armor, but the improved ballistics of sub-caliber ammunition is also of great importance in that it makes it easier for the gunner to hit at greater range.
Anyway, back to squishy humans. You are asking: can you use DU to increase the "punch" of a weapon so that weapons that can be lugged around by humans become a viable threat to armor?
DU is not a magic force multiplier. You still need to impart kinetic energy on the bullet. As this happens you will create recoil. And if you are trying to brute force your way though armor, you will create so much recoil that a human cannot handle it. This means you cannot increase the weapon's effect much by using DU.
Statically mounted heavy weapons already pack more "oomph" than your fictional soldier-carried weapons do. And then there are rocket propelled grenades, and anti-armor missiles, which do not rely on brute force but instead on clever explosives to make their way through the armor. Not to mention that missiles are guided and thus can be fired far outside the engagement envelope for your ballistic weapon. The thing is that modern armor is designed with these kinds of weapons in mind. And since these weapons pack more of a punch than that of your fictional firearm it makes your weapon moot because it does not add anything of value.
Then — as a bonus — you have the problem of the metal being slightly radioactive which will create all sorts of "fun" problems. No, not acute radiation sickness, but let us just say that having the common soldiery run around on the battlefield with fissile material will make the world community raise an eyebrow or two. Not to mention that their travels home after the war will be "interesting" when they set of airport, port and border alarms left and right.