There are a number of issues here.
Ok, so let's say you create a new kind of cone that's sensitive to UV. Where are you going to put it? The retina is already jam-packed with cones, so you need to remove other cones to fit the new cones. So your guys can see UV, but their sensitivity to one or more other color channels gets worse.
Now, if these are cybernetics or similar, you could potentially miniaturize the new cones and old cones and solve this issue. Maybe even give your guys better resolution than normal.
Another issue is that IR bands aren't physically capable of high-resolution imaging relative to visible light. The deeper into IR, the worse the image gets. On the other side, UV gets better resolution to an extent.
Sunlight generates a good deal of near IR, but it falls off exponentially, so you're down to 12% (ish) by the 1500 nm range.
You can use longer wavelengths, but you'd need to make the cones very sensitive in that band. Maybe you could center the cone's sensitivity in deep IR with exponential falloff away from the peak. The exponential gain in sunlight towards near IR would be offset by exponential falloff in cone sensitivity, potentially giving you good low-light vision across a broad spectrum.
If you go to far IR (8-15 µm, 8000 to 15000 nm), you'd be able to see objects at room temperature, although that does include your own eyeball which could get awkward. You'd likely need some kind of specialty cooling system to keep the lens, retina, etc. cooler than whatever you're trying to look at. Not sure how viable such a cooling system is with biology, although it's perfectly plausible with cybernetics.
At the other end, note that far UV is absorbed by the atmosphere and is damaging at a molecular level, so there's a physical limitation on how deep you can go into UV (200-300 nm).
There are a couple ways to go about this. First, you can replace the existing red/blue cones with new cones that have a broader response curve. So now an object that reflects IR becomes red, or more red than before. Same thing applies to UV objects looking blue or violet (depending on which cones you alter for UV response). You could potentially replace just one cone (say green) with a really wide response to both ends, but I'm not sure it would be beneficial.
Second, you can add new cones in the new ranges. This gives you much better control over which spectrum you can cover, and probably gives you better light absorption (most materials are crap at absorbing a huge range of wavelengths for photoelectric effects, although multi-junction cells might help here).
Now, there are two sub-options here. The simplest way is to attach these new cones to existing nerve outputs. So you'd see IR as red, UV as blue, like before. (Or IR as blue and UV as green, or whatever floats your boat. Again, not sure there's any particular reason to do it, but you might find that it helps night vision or something.)
The other option is to generate new nerve signals. This also requires rewiring the brain to accept these new signals. Obviously, it's possible, but I have no idea how hard it would be, or if it could reasonably be done on an adult.
If it worked though, the person would have a vastly increased color space. The difference between red and IR would be blatantly obvious to these people, along with the difference between blue/violet and UV. There would also be a difference to them between green, and green with IR, or green with UV, or green with IR and UV.
There would be 1 null color ("true" black), 5 primary colors (one for each cone), 10 secondary colors (each combination of two cones), 10 tertiary colors (each combination of three cones, which is also the combination of absences of cones), 5 quaternary colors (each combination of four cones, or absence of 1 cone), and 1 everything color ("true" white). Plus all the trillions of intermediate colors. I took the liberty of naming them and coming up with tentative pronunciations. Those aren't ANSI standard naming conventions or anything.
The specifics of what real-world objects translate to what pentachromatic colors depends a lot on exactly what response curves you use. Also, it's possible to move the existing cones so your guys' "red" wouldn't correspond to normal red.
For example, you could have IR equate to far-ish IR, Red equate to near IR, Green equate to red/green, Blue equate to green/blue, and UV equate to UV. This gives you a really broad spectral range, but you lose a lot of human color response. To normal people, you'd seem red-green color blind.