There are various gasses which are more or less inert, but they're a lot less common in the universe than nitrogen. Argon is a good alternative if your atmospheric pressure is fairly Earth-like, but it can become unhealthy at higher pressures (causing an effect similar to nitrogen narcosis, but occuring at much lower partial pressures). Neon has no such problem, but is rarer still on Earth, though more common than nitrogen in the rest of the solar system and sider Universe. Divers also use helium instead of nitrogen to avoid nitrogen narcosis] at depth, but having a lot of helium in a planetary atmosphere is problematic for other reasons (because of the additional gravity required to keep it from flying away into space).
You could replace nitrogen in its entirety with argon or neon with no immediate health issues, though there would be other interesting knock-on effects. Firstly, neon is somewhat easier to ionise than nitrogen (hence the common occurence of neon lights) so lightning would be perhaps more frequent and much more red than blue.
More importantly though, your planetary nitrogen cycle would have to be radically different... no nitrogen fixing, as there's little atmospheric nitrogen, and nitrogen gas produced by denitrifying organisms would be effectively lost to the system which would be extremely problematic. You could handwave some other reservoir of nitrogen, but preventing it from being released as gas over geological timescales seems like it would be quite difficult. Without a nitrogen cycle, life on your planet will have to be radically different to Earth.
More broadly, if you wanted to remove some or all nitrogen from all of your universe, not just a planetary atmosphere, you risk messing up the C-N-O fusion cycle which occurs in larger stars, or having to perform implausible handwaves to reduce its comsic occurrence. Probably best not to do that.