I've started this concept of a series of stories, in which an immortal being causes all of the conflicts for his own entertainment. The question is how could he go unnoticed by any modern government? Doing what I stated above would essentially by impossible to do without getting caught pretty quickly. It also doesn't work if governments know about him in general, because then they can prevent any/all chaos caused by him. Anybody have any ways around this?
Although you haven't explicitly said it, going by clues in your post such as 'modern government', I assume we're speaking of a human.
First of all, if you haven't watched the movie The Man from Earth, I recommend that you watch it. While it's not based on science, it's a fun play around the same idea, where a caveman from around 15,000 years ago, who simply doesn't die of old age. He simply lives a nomad life, has met Buddha, and also happens to be the inspiration for the Jesus story. He lives to today's date, and changes identity often and moves places.
Now, thinking about a similar person, we have to establish few things. By immortal I suppose you mean will not die of natural causes. It would be pretty hard to go unnoticed for millennia if you had Superman like powers where you're indestructible. In fact, such a person will have all the tools required to be the Supreme Leader of the Earth, if they wished.
Next, we'll assume this is not a power that's gained recently, and so the person has lived through much of the human history. Very similar to the idea explored in the movie above.
Next, we'd assume the person 'looks' to be in their 20s/30s; i.e., an adult but not quite old.
So, going unnoticed would have been fairly easy up until the 20th Century. Technology, particularly surveillance, was minimal, traveling from country to country was much more easy (in the sense many countries had much less strict immigration policies), instant communication wasn't available, the world wasn't connected. All you had to do was simply move far away enough from where you live. Only thing is you shouldn't stay in one place too long as people will notice you not aging. Someone in their 30s can pass for 20s, and even early 40s, but not beyond that. So basically, live somewhere for 10-15 years, pack up and move.
The game would change drastically after the WWII, with countries spying on each other (not to mention their own citizens), and identification of people is common. Things would get a hundredfold difficult after 9/11. That is the point in time where governments got serious about mass surveillance (read about Snowden), and tracking people nowadays is quite easy.
In the USA, government legally keep track of people by Social Security, not name, and I'd assume most countries have a similar system. So you'd have to swap identity every couple of decades and get a new social security number etc. This is not entirely difficult if you know the right people, and given that our subject has lived through millennia we can assume that they know how to do this. But that won't help you much getting past modern day mass surveillance. It doesn't matter where you live, powerful countries like the US, China, UK, and many modern developed world countries can track you wherever you are.
It doesn't matter where in the world you are from, but you'd be tracked. Unless you live a completely offline life, which would be much easier in a really rural part of a 3rd world country where there isn't much access to internet. But then living in such a place won't probably give you opportunities to 'cause havoc' as you put it.
So, I'm afraid that your 'hero' is going to be fine till late 90s at best, but after the new millennium, they're going to inevitably get noticed.
He uses contacts to swap his identity to a younger version every forty of so years
An immortal person has money. Lots of it. Place a few hundred dollars in a bank, let the interest accrue for a few hundreds years, and voila! Instant fortune, minus the instant. Using this fortune, its entirely possible to do things like set up trusts and grants, all controlled through dummy corporations or the like, and then he uses that to gain contacts.
How? Perhaps a few well-placed bribes, perhaps grants to poor students to bring them into power, perhaps he's a ranking member of a cult - there are myriads of ways to assemble followers. At that point, he simply greases the wheels of his engine when he needs to, whips up a new identity and 'dies', leaving his assets to his new 'younger' self.