You have the right idea, but misplaced. All that is required is that the island be the peak of a seamount which rises steeply from the ocean bottom. Tsunamis in open ocean have little amplitude (ships hardly notice them), but extremely long wavelengths. When these approach a shallow, sloping coast the water in the fromt of the wave starts to slow down, and the rear part catches up, the velocity of the wave increases and water starts piling up into a much higher, shorter wave. Coasts with a steep underwater approach simply don't get the "tidal wave" effect.
Another way to look at it is that, since a shallow approach produces a short, high, fast wave, the wave has no chance to move laterally around an obstacle, and tries to go over it. A longer, slower wave has the time to be deflected around an obstacle without much amplitude increase.
EDIT - In comment, the question was asked, "what do you mean by "the velocity of the wave increases", and a response will be too long for a comment, so I add it here. The short version is that I misspoke - the velocity does not increase.
As the wave propagates into shallower water, it loses velocity (proportional to the square root of water depth), and gets higher.
As its height increases, though, at some point the wave shape changes from being part of a wave system to that of a solitary wave - basically, once the wave trough touches the sea bed the behavior of the wave changes radically. I misspoke in saying that the wave speeds up. It does not. What I meant was that the higher the wave, the faster it propagates once it becomes a solitary wave. It may help to consider that a wave is in constant collapse, as the weight of the total wave causes the lower sections of the wave to squirt forward. The higher the wave the greater the pressure and the faster the base of the wave moves - and the dynamics of the wave as a whole keep the upper section in place, although the actual water molecules are constantly recycling their positions within the wave. And, of course, at this point the wave will begin dissipating as it travels inland.
If you ever watch one of those documentaries about the crazies who surf the Condition Black (40 and 50 foot) waves in Hawaii, you'll notice that all those surfers get carried out on jet skis and start their runs behind the jet skis. That is, they don't get dropped off, stationary, at the base of the wave. There's a good reason for this. If they are stationary in the water, the wave moves so fast they they can't accelerate fast enough to keep up with the surfable portion of the wave face, and will either fall down the vertical portion or get left behind by the wave, depending on the exact shape of the wave. Mostly, they fall down the face and get driven under.