I'm sure the radiation would still persist at least a little after 5,000 years
Though you haven't specified the nature of the radioisotopes, lets have a quick look at real-world nuclear waste. Short and medium-lived radioisotopes will be mostly gone by 5000 years... Samarium-151 has a half-life of 85 years, for example, and after 58 half-lives there's going to be precious little of it left.
Long-lived isotopes will be the only things remaining. Technetium-99, for example, has a half-life of over 200000 years and so at ~98% of the original waste will still be present. (note, don't confuse it with Tc-99m which has a much shorter half-life)
On the flip side, long-lived radioisotopes are long lived precisely because they aren't super radioactive. You won't get the classic glowing green gloop or barren wastelands (unless the site was already a barren wasteland independent of the material stores there, I guess). There won't be any of the classic signs of radiation poisoning visible at all, I suspect, but more pernicious chronic illnesses that don't show up for a few months or years, or maybe not until you have children.
If humans live there, they probably have moved in comparatively recently as historically it would have been a hazardous place to live with high rates of birth defects and cancer and so on. Folk memory might have preserved knowledge of the history of the people who tried to live there, even if it didn't remember the origin of the site.
After 5000 years though, it might just be "less healthy" than uncontaminated landscapes, and it might potentially even be less dangerous than other kinds of toxic industrial waste that don't decay over time. Nonetheless, people who live there, or drink the water, or breath the dust, might well be marked with shorter lifespans due to elevated cancer risks and deaths due to leukemia or lung cancers will be quite graphic to observers (for want of a better term). Older animals there might have tumors visible when butchered.
Bad ground, tainted ground, evil ground... all sorts of superstitions. If illness is associated with artifact hunting then the artifacts themselves might be treated with great suspicion... if the "curse" laid upon the ancient tombs of our forebears takes years to strike down those who defile those tombs, who is going to dare keep looted goods in their homes? That's quite a gamble for anyone to take!