Most natural satellites in our Solar System are expected to contain fission furnaces at their cores, and some of these moons do have a striking resemblance to an asteroid. I'm wondering if we could establish a colony on an asteroid and harvest the "aster-thermal" energy (if there is a proper term for it) or rather create a space tourism hotspot for people to enjoy a relaxing bath in the hot spring. Is it possible for an asteroid to contain a molten core or do I need to look elsewhere to locate my "colony"?
Yeah, entirely possible.
The main issue is that because of their size, the ratio of surface area to volume is higher than the average planet. This means it cools much faster. So while some may have molten cores, they won't stay that way for as long as a planet might. Additionally, most aren't very near large gravity wells like a moon might be, so they don't have the same tidal forces adding energy to their interiors.
All this makes it less likely to find an asteroid with a molten core, but it's not impossible.
Ceres is the largest asteroid (or smallest dwarf planet, whatever); the core isn't molten based on best current data and modeling, but new data is forthcoming soon, in the sense of Earth, Io; but it may be warm enough to have liquid water. Note that Mars also doesn't have a molten core and that Io and the Moon, etc. have molten cores due not to nuclear decay or to primordial heat but gravitational energy primarily.