I'm going to push back a little at the numbers I'm seeing and say as few as 5,000. @Vincent's answer really opened up with the right answer - this depends on other things.
This may sound absurd, but count the number of ships you can dock at once. Then using the largest ships you could have docked, draw a walking path from the deepest hold in the vessel to the merchant's holding areas. Place one person about every 5 feet. This is how many people it takes to unload or load the ships at any one time, in the worst case. Certain kinds of loading requires not having that line of people there.
How many people live in this place is largely going to depend on two things- how many it needs, and how many actually want to live there. You said it was on an island at a river mouth. I could see that getting overcrowded, smelly, loud and generally unpleasant - if you wrote it that way, of course.
5,000 people is a lot to have in a small place, and depending on what they're doing, more is not always a great thing - they tend to end up working slower if they have to work on top of each other.
The point is, the fact that a place is important economically and strategically does not necessarily lead to it being popular. Other factors go into this, and a lot of it has to do with the personality of the persons in charge, and how that interacts with other people around them. If the lord of the realm has other land under his control, he may very well have decided that the trading port really is just that - a port. Having built only that which was necessary to service the port and the needs of the sovereign Whoever, other resources could have been used to improve other parts of the land.
Note that the 5000 number doesn't actually require that the place be awful - just that it's not sensational, and that it's not old. If people in the area are pretty sure that the continent they live on is the coolest thing ever, the world be damned. If it's been there quite a while, obviously, the population stands to have grown - peasants didn't just up and go somewhere else without good cause.
Please note - I am by no means suggesting that anyone here is wrong. But the staffing requirements of the port aren't as high as all that, and the rest depends on history and attitude.