It's pretty specific circumstances.
The sun is moving in an arc. Depending on year and latitude, it could be from less than 1° (just below the polar circle in midwinter) to 360° (polar summer).
Natives will see the rainbow if the sun goes over their heads into some particle cloud that refracts light.
So for the rainbow to last all day, the cloud needs to be an arc around the natives, as wide as the Sun's arc.
In theory, it would be enough if the cloud moved around the inhabitants. I can't think of a mechanism that would do that - maybe others can.
Barring such a mechanism, this limits the arc to a maximum of 180 degrees - at that point, you'd have some permanent cloud front and the natives living on the edge. Doesn't quite fit your requirement that it's a single location, the phenomenon would be visible along the whole edge.
So you're restricted to some area where the Sun is describing an arc considerably less than 180° - maybe 90°? And it would have to be winter so the sun isn't up for more than 12 hours and you can't have that "all day" phenomenon anymore.
With that, I arrive at this scenario:
There's a fog-filled area.
Accessible from just a single point, otherwise there will be too many points where you see the rainbow just for a short part of the day. A peninsula, maybe, and non-navigable waters at the border? Or maybe it's in a valley and accessible through just a single pass?
That point gets Sun all day (if the weather is right- there are regions where the weather is actually predictable), and the Sun goes up at the 4-o'clock position and down at the 8-o'clock position, with the valley or peninsula opening across 10-o'clock to 2-o'clock, with the rainbow going up, around and down with the Sun.
It's a pretty specific scenario, and it strongly depends on where the natives can move, i.e. their technological abilities - the whole thing will be totally demystified as soon as the first balloon goes into the area.
If you aim for a stationary rainbow, that would require a light source that's brighter than the Sun.
This could happen in scenarios where artifacts of a high-tech civilisation still function when lower-tech natives move in, but I doubt that's the scenario the question was asked for ;-)