Most of the radioactive/rare-earth that I saw are present in the form of silicates, hydroxides, oxides, phosphates, carbonates, borosilicates, vanadates. Haven't seen any in the forms of sulfides or arsenides. So probably no chemically toxic gases.
Radon has a half-life of only 3.8 days, you'd need a lot of uranium/thorium around to get a high enough concentration of radon (in which case, radon is likely the least of your problems)
If the mines are no longer operating, the particulate suspension in the air is going to be low. At the best, such exposures may occur if your character causes a collapse of some dust heaps (assuming dry dust accumulations)
I'd be more worried about exposures to water leeching the rare-earth/radioactive ores around and pooling them in concentrated slurries here and there.
why the area is not flooded
kaolinite does not evolve in always damp conditions
While studying soil formation on a basaltic rock in Kivu (Zaïre), they noted how the occurrence of kaolinite depended on the "degrée de drainage" of the area involved. A clear distinction was found between areas with good drainage (i.e., areas with a marked difference between wet and dry seasons) and those areas with poor drainage (i.e., perennially swampy areas). Only in the areas with distinct seasonal alternations between wet and dry was kaolinite found. The possible significance of alternating wet and dry conditions on the transition of allophane into kaolinite has been stressed by Tamura and Jackson (1953).31 The role of alternations between wetting and drying on the formation of kaolinite has also been noted by Moore (1964).32
One can plausibly assume the kaolin deposit was draining through cracks into the bedrock, so there will be at least seasonal opportunities to descend into the old mine galleries.