I want to set up a long-term self-sustaining colony on Mercury. What and how needs to be done for a colony to survive and thrive on the planet Mercury?
Blue Mars (from Kim Stanley Robinson's Mars trilogy) has another approach to consider: a city called Terminator rolls around Mercury's equator on tracks. It keeps pace with the rotation (thus the city's name as it is always located near the terminator). The motion is driven by the thermal expansion of the rails. See here.
All you need now is to figure out how to set up those rails and all structures that go along with it to withstand about 700 K of surface temperature (here). Technical ceramics might do.
Go deep underground. Old theories had Mercury tidally locked to the sun (one face always facing it, another always facing away). Under that concept, the best place to live on mercury is on the dark side, with heaters. The current concept has Mercury slowly rotating its face relative to the sun. Sunside heat would be a quick death for humans on the surface unless you used some handwavium force fields to shield you from radiation.
Going deep underground allows the surface to act as a radiation screen, which would simplify the overheating problem. You would still need to have a way of removing excess heat though. Possibly by importing and venting liquefied gasses to vacuum, relying on the phase change of the gas to cool things down. Any resupply ships (carrying cooling gasses, for example), would need to land at your base when it was on the side away from the sun, or have heat problems as well.
Mark Ripley got half of it.
Yes, you go underground. However, by choosing your location properly you don't need air conditioning. The poles of Mercury are actually cold, there is a line around them (I forget the exact latitude) where the average temperature is comfortable for humans. If you dig deep enough at that point you have a place that will stay comfortable all the time.
In practice you will want to be located a bit closer to the poles as you will no doubt be putting heat-producing equipment in your habitat. Balance your latitude against the heat load of your equipment (and bodies) and you still need no climate control.
Put solar panels up on supports so they're pointed directly at the sun, use the power to run lights for your farming that provides food and oxygen.
Mercury has extreme temperature. The side facing the sun is extremely hot (around 400 °C). The best place to live would be near the boundary of these regions, a place where the time is sunset or sunrise. A good idea would be to stay in a very large trailer so that you can drive and keep moving to keep the sun at the horizon.
Mercury's atmosphere is extremely thin. Almost non-existent. In order to survive there, you would need to carry your air supply with you.
Having no atmosphere also means that you will have to keep wearing your space suit. Otherwise your blood will start boiling within seconds and evaporate away quickly, killing you by dehydration.
Mercury is either too hot, or too cold for any sort of Earthly plants. This means that normally you won't be able to grow crops. But if you go underground at least 30 feet and make an 8 feet hole in the roof, you will have enough sunlight to grow crops in a 10x10 meters area. You will also need Earth's soil for that. which you would need to bring with you on your spaceship.
There are craters on the poles of mercury that are always dark and have water ice. There may also be in the poles some peaks of eternal light, at least nearly so. This means that a colony on Mercury is possible by being in a dark crater with the water that is needed to support the colony already found there. Energy would not be a problem for such a colony, radiation protection would be.
In looking for sources I found this which reaches the same conclusions as me.