Only a small number of these tanks are going to be produced, so cost isn't really an issue. They are light and fast, but need to be able to take out another light tank at a long range with a few shots. The technology level of my world is not much above that of Earth now (no unobtainium or super-mega-blaster lasers). Man-to-man conflict is not common at all, so something like a machine gun would be impractical.
In general, take a look at IFVs (Infantry Fighting Vehicles, the things that are responsible for "Mechanised" in "Mechanised Infantry"), and work from there. Since you want "tank", you don't need compartment to carry infantry around, so you can scale them down.
Bushmaster 25mm autocannon.
Bradley IFV, armed with Bushmaster.
Bofors, designed before WWII, originally as Anti-Aircraft weapon.
Bofors on modern platform, the Swedish CV90 IFV.
Or perhaps for maximum "rule of cool" the GAU-8 Avenger
Best known for having this plane attached as accessory.
In general, you are looking at penetration of 50mm RHA (Rolling Homogeneous Armour - armour thickness is stated in equivalent of rolled steel nowadays, instead of raw thickness) at 1km on those weapons with fairly modern ammunition. If you are placing that in near future, you can claim further advancements, and increase that.
Size of IFVs shown above is comparable to modern MBTs (Main Battle Tanks), but that's because they need to carry ~10 fully equipped infantrymen. Remove this requirement and size drops, which decreases volume requiring protection, which means lower mass of armour for same protection, and thus lower weight. Those example IFVs have about 30mm RHA armour.
For reference: modern MBTs have frontal armour ~800 RHA, which is why they use completely different weaponry. If you are against something with IFV grade armour, then more than 40mm gun is complete overkill, both 125mm, 120mm, 90mm or even 75mm will be too much, needlessly increasing weight, length, stabilisation requirements, and make it harder to include autofire mode. Autocannons can pack plenty of ammunition, and shoot a lot of it, some go as low as 90 rounds per minute, rotary ones go as high as 10000.
To elaborate a bit on a comment under another answer
Take a look at T-14 Armata, latest Russian MBT. Yes, it's an MBT, but it's the first MBT to use auto loaded remote controlled 125mm cannon with option of computer controlled autonomous fire (Russians have been using MTB autoloaders for decades now, but it's the first time when next logical step was taken and remote computer fire control was added). Crew of 3 lies (seats are heavily slanted, so that crew is in position between sitting and laying, this is done to decrease height of the compartment and increase comfort on prolonged missions) in separated armoured compartment in hull, while turret sacrifices a lot of armour (MBTs with crew in turret need 800 RHA both on front of turret and front of hull) because hit in the turret may damage or destroy the gun, but platform and crew will survive to retreat for repairs. Because of that, Armata weights 15 t less than western tanks in service today (~50t versus ~65t, IIRC among the reasons is fact that Russians couldn't figure out how to make engine and transmission durable enough to reliably propel 65t monster at required speed).
If you want just crew of one, you should be able to mount 30/40mm autocannon, armour up crew bathtub (to borrow term from A-10) to 150 RHA to make sure operator can't be killed by 37mm guns in worst case scenario, have 40 RHA on rest of vehicle as first line of protection (keep in mind that slope increases effective armour so front with nominal 40RHA can be much thicker when fired upon, but this is a very complex issue, and a lot of considerations enter into designing ammo which can decrease angle of attack, and armour which can counter those ammo design tricks), and fit that all under 20t. To use CV90 (because IMO it looks very cool, don't you agree?) as example, latest versions weight 35 tonnes and 810 HP engine, giving it HP/t of 24 which is on par with MBTs, if you used same power plant on 20t vehicle, you would reach 40 HP/t, for military vehicle this is actually a lot.
At the same time, walker robots would have to have worse armour, for simple reason your tracked robot killer has better volume to surface ratio, and has much more room for engine, fire control and gun per unit of armour. Without having to armour joints and lower profile, it could be very resistant to the exact same type gun which it could use against them.
I would consider mounting long range guided anti-tank missiles on the light tank, similar to those carried by the M1134 Anti-Tank Vehicle. These have a much longer range than a 37mm cannon and would enable your light Tanks to destroy the enemy before they could engage effectively.
If we assume that human soldiers aren't a major concern, and that technology is similar (perhaps a bit above) modern day earth, I would go for a major revision of the whole tank concept. The thick armor is meant to protect the crew, stop enemy tank shells from penetrating, provide cover for friendly troops, and protect from mines. If you made it a drone tank you could make it super tiny, fast and hard to hit with conventional munitions, thus ditching most of the armor.
Then, load it up with anti-tank missiles. These would let it take out enemies from great distances without moving in range of their cannons. Flak missile defense systems could protect it from enemy missiles, and it would need some kind of system to spot and avoid (or trigger) mines.
So you seem to have two requirements: you want to 'one-shot, one-kill' an enemy tank at a relatively long range, and you want to be protected from a 37mm gun on an evil (I assume) tripod robot.
The 37mm WWII type guns (I'm not really aware of any modern varieties) like the American M3 met its match by 1943 with the late-model Panzer III (with 30mm armor) and Panzer IV (80/30/20mm, front/side/rear) in North Africa. So we can assume that we want our armor in the 30mm range, possibly less to account for modern advances.
This makes me think of the Kanonenjagdpanzer; slightly more lightly armored in the 50/30/10mm range, but with a 90mm main gun that should put a hurting on anything, especially anything that might call itself a 'light' tank.
This tank is 'light' in the sense that it is only half the mass of a modern main battle tank, but it is still 25 tons+. It has good mobility, and a low profile to help it defensively, but without the swiveling turret it is poorly defended against infantry. However, you suggested this is not a problem. Depending on the speed of the enemy evil tripod robots, the casemate design (e.g. without the turret) may or may not be sufficient to hit them, but in any case, if the robots only have a 37mm main gun, they shouldn't be too threatening at medium to long ranges.
Generally speaking, an ideal weapon should have the highest probablity of destroying your opponent in a given period of time. Ammunition supply is another consideration. If your opponents are very lightly armored, a high-caliber machine gun would do. You don't need any artillry or rocketry in that case. If your opponents are armored, but lighter than a tank, you would need a fast-firing cannon similar to the ones used in warplanes. If your opponents are armored similarly to your tanks, you would need a higher caliber cannons and rockets, just like modern tanks have today. If your opponennts use active defence systems, then you would need to use more sophisticated missiles or maybe even go for more exotic weapons like lasers or EMP-based ones.
P.S. The above answer assumes that your tanks will be sufficiently armored against 37mm guns. If not, your ideal weapon should have a range that exceeds the range of your opponents, which means that machine guns would not likely be a viable option.
If we are considering this for a near-future warfare scenario where infantry are not the primary consideration (you may want to re-think that. There is no possible war scenario where infantry is not the primary component of all forces). I would get a little bit "creative" here and try something slightly exotic, especially since you said cost is not a concern.
I might be tempted by a Metal Storm type system. Obviously, this would not be anti-infantry caliber like most of the Metal Storm prototypes, but would be something in the 20-30mm range. Such a weapon would be able to slice through most other types of vehicle armor and would be deadly against aircraft (unless the aircraft were using stand off weapons outside the vehicle's range).
If it has to worry about heavy armored opponents, I would just go crazy and use a miniaturized rail gun. We have to assume "near future" tech for this, because you need to generate a LOT of power to shoot one of these things, but it is entirely plausible that what can currently be mounted on a big ship could be mounted to a smaller vehicle in the near future. The vehicle would clearly need some highly efficient power generation, and might logically be a hybrid or electric drive because it already has tons of electricity available anyway. Railguns have some issues with rail wear, but that is something that can probably be worked out with better design and materials.
The upside is that you can propel a projectile at tremendous speed and such a projectile could be something like a tungsten or DU (depleted uranium) rod used to penetrate MBT armor in current generation discarding sabot rounds. If such weapons are practical on a medium armored vehicle, MBTs might be less common due to the fact that the extra armor doesn't save them, but will still slow them down. In such a scenario, you might have medium weight vehicles maximized for mobility and stealth taking on some of the roles currently reserved for tanks.
The role of the scout tank has been obsolete for a while. It pretty much got replaced by the attack helicopter in the 70s-80s. More recently, this role has been given over to the UAV. The closest thing we have in modern time to a scout tank is the APC- basically an armored troop hauler with an autocannon and some missiles. But these aren't scout tanks and they don't do scouting.
Traditionally, scout tanks (M3 Stuart, AMX 13, etc) have treads (for mobility), just enough armor to protect against small arms (less weight = more mobility) and just enough gun to fight back against infantry and light vehicles, supplemented by anti-tank missiles in modern vehicles (ie, APCs like the Bradley, BTR, BMP, etc). Scout tanks traditionally sacrifice firepower and armor in the name of saving weight to increase mobility. But they are still quite heavy because they have to be resistant to infantry weapons, not to mention IEDs and RPGs.
Traditionally, the light tank formula has resulted in vehicles that have good mobility for a tank, but very poor mobility in comparison to everything else, especially flying vehicles. And because they give up so much armor and firepower for that mobility, they are tend to be vulnerable to IEDs and handheld anti-tank weapons, not to mention the guns mounted on actual tanks or the anti-tank missiles mounted on vehicles. They're also nearly useless against real tanks and fortifications because big guns are heavy and you can only mount so many big missiles on the side of a small tank turret.
A helicopter or UAV is much faster and harder to detect (better for scouting), and since it can't be hit by small arms or IEDs in the first place, all of the armor can be deleted in favor of better sensors and more missiles. And since UAVs aren't manned, you don't have to train up a new tank crew every time you lose one.