NOTE I have re-edited the title to make it clearer. I have also added some detail in the paragraphs below that does not invalidate any of the existing answers.
Let us suppose that a group of humans several thousand years ago were shipwrecked on a barren volcanic island. The only living things they brought with them were potatoes both for food and for seeding. These are ordinary potatoes as we know them. There are no other vegetables or animals on the island.
The original settlers had fishing equipment but over the centuries this has disintegrated. Growing potatoes has given them food plus fibres from the roots, stems and leaves. They can fashion simple tools from seashells and fish bones, etc.
There is no metal ore to be found so they are limited in what they can make. Nevertheless they are intelligent and resourceful and invent as much as they can using potato and sea products. There has been absolutely no contact with the outside word.
Is it possible for a population to survive for this long? Can they live on just fish, potatoes and the occasional sea bird? Can they build shelters from potatoes? Can they, given time, even build boats from potatoes?
Question
What are the limits to potato living?
Please ask for any necessary clarifications.
Assumptions
The climate is generally moderate with the occasional severe storms.
The original settlers did bring some artefacts with them but nothing remains that is of practical use.
They have a constant supply of clean fresh water from a stream.
The ancestors brought fire with them and have faithfully kept it going all this time, keeping it alive with potato leaves and roots.
Potatoes grow best in loose, well-drained loam soil. Heavy clay soil retains too much moisture and tends to become hard as it dries, which can eventually slow water absorption and make it difficult for tubers to grow. Sandy soil drains too quickly, which can result in drought stress unless you water often. (Source).
The island is protected nearly all round by a reef. The fish are anything that might have been found in the healthier days of Australia's Great Barrier Reef.
The rocks are mainly volcanic.
They can make fibres from potatoes and/or their own hair.