It might depend on if your natives are human and just live there over several generations, or if they took a different evolutionary track.
On earth, being able to pick out camouflaged creatures is actually EASIER if you are colorblind. This is the reason why South American primates tend to be colorblind.
It might not just be about lighting either. It might be about environment. Here on earth green environments are the norm, so we adjusted for that.
But on your blue sun planet, it might be a different story:
Because of this, natives from the yellow sun system may have different advantages than blue when they switch worlds. It might be possible that one or the other may be considered to be "colorblind" on the other planet, and this may confer advantages and disadvantages respectively.
The races might be genetically compatible, but even here on earth, there are differences from person to person. And it's been postulated that we don't even see color the same way from person to person.
For example, you and I are taught that a particular shade is "red." We may, in fact see slightly different shades, but since we both know THAT shade as red, we both call it the same thing.
When it comes to red-green color blindness, it's more marked. For hunters tracking prey, not being able to see the difference actually makes those who are colorblind in that way better at picking out animals. This may be why, if men were the main hunters and women the main gatherers, more men experience colorblindness of this kind, and more women can see differences in shades (because this was more important when it came to judging ripeness when gathering).
Those on the blue planet might even see differences in color that the yellow planet folks can't and vice versa--so natives of either may consider most of the population of the other planet colorblind. Because there is a lot more to the spectrum than we can see, and there are even animals on this planet that do not see what we can, but can see colors we cannot readily perceive.
While you may believe that because they are genetically compatible that your two populations will see colors in the same manner, consider this fact: within our own population there are differences in how we see color, even when people are genetically compatible. Take that to two different planets, and there may be even more of a difference.