Depending on the sea conditions, you might be able to do something interesting with waves. Ignoring wind and air resistance for a moment, a craft like this will tend to slide down a wave, so if you are going the same way as the waves then propulsion is very simple. If you combine this with a sail then you could, if you are clever enough, use the difference in wave and wind movement to control your direction.
The Wandering Albatross does something like this to glide without flapping its wings. Your islanders are not going to do exactly that, but they will probably have related techniques. The simplest thing would be to use a sail to move sideways along a wave, using the trough of the wave a bit like a keep, thereby going to the left or right of the wind and wave direction. I don't think you could tack like this though.