Is it possible for people to carve tunnels and caves into a mountain that used to be a volcano?

This volcano has not erupted for several thousand years, so is considered extinct by all civilisation. The carved-out volcano/mountain is used as a temple of some sort. The society that lives around the area now have Medieval-Renaissance level technology and small access to elemental magic.

Will the extinct volcano be made of rock or dirt or have streaks of obsidian running through it? What would be the conditions inside any tunnels/caves carved into this ex-volcanic mountain if it is at all feasible to do so?

EDIT: I assume that this possible. Ideally an answer will explain what the possible (or likely) geological composition is and the possibilities of how to deal this with Medieval-Renaissance tech.