In this medieval-fantasy (Tolkien-esque, except humans are the only race except for some incredibly rare magical ones) world, a few individuals ( have become mages, sorcerers and monks - which are all in-universe terms for the same thing - by using precious stones to channel the magical abilities inherent in every human. The powers available to the magic user vary depending on the inherent elemental connection of the stone; for example, someone with a ruby ring could manipulate fire, or someone with an opal necklace could control water. (A magic user could collect many gemstones and thus use a plethora of powers.)
Because of the relative rarity of the gemstones necessary to wield magic, the number of powerful or skilled mages is very low. This means that mage-on-mage battles are ultra-rare, the stuff of myths.
The mages can use magic to teleport themselves indefinite distances, with enough practice. This is more dangerous over larger distances, with less magical power (smaller gemstones) and for less skilled magic users, and if you only have a rough picture of where you want to go (i.e. you've never been there). It would be easy to mess up - but if you do mess it up, you don't get pulled apart like Harry Potter wizards/witches. Instead you get teleported to another universe. You can get back with enough tries, of course, provided you don't lose your gemstone or get eaten by the weird inter-dimensional oozes.
Any mage could teleport inside the treasury of a famous ruler, grab some gold, and teleport out again without anyone being aware except for the mage(s) working for that ruler. And even the most skilled mage, ever, would not be able to stop someone unless they spent all their time in the treasury, waiting for thieves.
Naturally this makes mages incredibly powerful. Not only are they practically impossible to capture, except by another mage, they can also go anywhere and get anything. In fact the power of a mage who has practised dwarfs every non-mage character.
How can I limit their teleportation while still allowing them to go anywhere with little risk?