Quasars:
There are indeed landmarks in outer space. Quasars are very bright(at their source) but very distant objects, visible from effectively the edge of the universe. So distant than any local (or even intergalactic) travel will not distort their configuration.
There are not so many that one can get lost among them, yet there are enough that they provide a very nice 3-d orientation of space for the observer.
OK. So we know what direction is where. This still does not tell us where we are.
We need other, closer reference points. Fortunately, we have them:
- The core of our Milky Way Galaxy
- several nearby galaxies.
With the absolute orientation provided by the Quasars, and the direction of the galactic core, and the relative offset of nearby galaxies relative to the core, we can determine our position accurate to within single-digit parsecs anywhere within the Galaxy. The only reason we cannot get even better accuracy, down to a few AU, is that neither the galactic core nor nearby galaxies are point-sources. We need to estimate within a small fraction of a degree where their "middle" is, and this introduces a minute bit of fuzzyness to the measurements.
Any better navigation than this will require star-charts of the appropriate region, of course.
The only time when celestial navigation such as this would fail, is when the optical view of the surrounding space is obscured. I.e. in a very dense nebula, etc.
But if you can see the sky, see the stars.. You can pinpoint your location.
On Pulsars: People love to propose using pulsars for navigation.
Pulsars are Neutron stars that emit pulsed radiofrequency signals as they rotate, with extreme precision. So one would naturally want to be use them for navigation.
And if you are limited to sublight speeds, it will work fine. You will never go far enough, and never bypass time by traveling too fast, for these signals to deviate. Good, pinpoint, loud, radio beacons!
But in an FTL-capable universe, they become worse than useless.
The problem is that a pulsar rapidly slows down the rate of spin.
For a newly-made pulsar with a pulse rate of 1-10ms, the slowdown rate p-dot is at a rate of $10^{-10}$ seconds per second. This means that every about 120 days the pulse rate slows by 1 ms. For a slower Pulsar in the 30ms range the p-dot is about $10^{-12}$ to $10^{-13}$, after which slowdown remains roughly constant until Pulsar "death", where the magnetic field is so weak that the pulses become weak and fuzzy. A 30ms pulsar will slowdown to 60ms in about 1000-10000 years.
If you fly at FTL speeds towards a pulsar, you will see "newer" light from the pulsar. In effect, the pulsar will appear older, and slower. As you could only recognize it by the rate of its pulsing, it will now appear to be a different pulsar, and you will be lost in space!