The situation is that two magic users are fighting a concrete golem, their only way to defeat it is to destroy it, when this characters crack it they find that the golem is hollow so their strategy is to fill the golem with water and heat it until it explodes because of the pressure. How hard would it be in a cold climate with the last air bender type of fire bending?
Not hard at all, provided that enough heat at a sufficiently high temperature can be supplied. Pressure will simply increase until it exceeds the concrete's limits. Note that while concrete is very resistant to compression, it is not resistant at all to expansion, which is why pre-stressed steel reinforcements are employed structurally. To be able to move, this golem cannot have much in the way of internal reinforcements.
Saltwater exposed to lava temperatures (1200 °C, but may be as low as 800 - 900 °C) is known to cause explosions powerful enough to crack solid granite ("littoral explosions"). If the steam does not find a release, imprisoned water can hurl shrapnel up to 3 miles away.
Also, it is documented that the reinforced vessel of a nuclear reactor core cannot withstand a powerful enough steam explosion.
Thick concrete is a decent thermal insulator, so the cold outside won't be able to cool the water inside quickly enough.
So, I am confident that the golem won't stand a chance.
If the golem is thick enough to work a good head of steam, and they're too near the golem when it goes kaboom, neither will your heroes.
Kill a concrete golem. You have: water, heat.
If heat can be added magically, presumably it can also be removed magically. So, it might be possible to encase the golem in ice.
Otherwise, humid vapour can quickly enter microscopic cracks in the concrete, and can then freeze, expanding and causing spalling. This ought to be done several times to damage the golem, unless some points (e.g. the joints) are more vulnerable. Moving a joint while locked in frozen water should impart considerable surface damage, reducing mobility.
Also, adding a large quantity of water should alter the balance of the golem, and possibly even make it fall down.
Water has a very high specific heat, meaning that it takes a lot of energy transferred to water to increase its temperature. To rapidly heat water, it will take a large amount of energy, and in the case of heating through flame, a hot flame with long exposure will be required.
Concrete has the advantage of very a low thermal conductivity meaning heating the outside of the concrete Golem will not heat the inside very rapidly.
If the temperatures of the flames are hot enough to heat the water inside the Golem rapidly enough to cause an explosion (there's also the problem of pressure since any holes in the Golem would make this even more difficult), they would be hot enough to destroy the concrete on their own which would create cracks as the water is heating.
If possible it would seem to be a better strategy to first heat the Golem and damage the concrete, then fill it with water and freeze it. Water expands when frozen and this would be destroy the structural integrity of the concrete and be much more realistic than heating the water rapidly enough to cause an explosion.
Why would a concrete golem be hollow? Use that against it!
It is tricky to cast a hollow thing out of concrete - much easier to make something solid. They don't really need to know exactly why it has to be hollow, but they want to find out. By making it not hollow.
Fill the golem with concrete, not water.
Maybe non hollow golem does not work. Joints don't bend? Too heavy to move? Then you are done.
Maybe solid concrete makes concrete golem not work. Then it will work less and less as concrete sets up.
Concrete expands as it hardens. Maybe you will blow up golem after all?
Golem is cracked already - that's how they know it is hollow. Water might drain out crack. But cement is thicker and hopefully will stay in crack.
How exactly you fill a hostile golem with water, concrete, Vegemite or anything else is left as an exercise for your characters.