I think that you are in the beginning stages as it seems as if you still don't have anything solid.
My thoughts are that you first need to determine if anyone knows that they exist. In X-Men, their existence is know and some can pass in everyday life without anyone knowing about them. In Harry Potter, they aren't necessarily weak; however, they have been persecuted in the past which leads to their desire to hide their society from muggle knowledge. So, to start off, do their powers give them the ability to establish a society completely hidden from the modern world or will they be dependent upon the modern world to survive.
Next, I think you will need to determine what happens if an individual if found out. In X-Men, it's kinda like coming out of the closet...can't exactly shut the door while Harry Potter has a backup plan (that gets used a lot--possibly overused--would have been a real bummer if using a memory modification charm on someone who had a blocked memory already caused the earlier modification to be undone).
If you have gone with the X-Men route, you ideally want to pick a country that is more "chill" and not as opinionated. In the U.S., you have a highly opinionated society that is ruled by a majority (within Constitutional limitations of course). In Africa, you have very weak governments that might not pose much of an issue, but you also have a weak infrastructure and a high degree of taboos (and people are willing to kill over it in some cases). Regardless, you might be better off within the developed world where some degree of law might offer some protections. Your biggest threat will be members of your own society and more specifically, the few whose actions are in the minority that push the more extreme members of the general public to hate you even more.
If you chose the Harry Potter route, then you could pick anywhere really. They are self-sufficient capable of hiding themselves from the world. In this case, your biggest threats also come from within. The threat of exposure is the greatest and it can be achieved by a single person In a population of 50,000, this is extremely easy if you are not isolated and is an eventuality should you have no way to combat it.
Regardless of your path, I think you should also come up with a list of certain powers that might be helpful in the establishment of your society. It's hard for us to give you adequate help as that will play a major role. If mind/memory manipulation is possible, many doors open for you and if not, then they close. Also, things like power nullification (useful against others with powers) would be extremely beneficial when used in law enforcement and maintaining secrecy when in the right hands (but a detriment in the wrong hands). So overall, I think you need to make the determination I suggested above and then figure out some key powers that will be necessary for your civilization to be established.