To survive in the long run, your population's diet will have to diversify in range, shrink in quality, and vary hugely in quantity
Their diet on land will expand to encompass a wider definition of what animals count as food than most of your modern readers, 'pest' species included: rats, bushtail possums (if we're using the modern New Zealand example of Mahia as a baseline), dogs, deer, feral pigs (who will in turn eat the human's waste) and so on. If it moves on or near land, and it's not horribly poisonous or venomous, they will try to eat it.
Being on a peninsula, they will have access to nesting or burrowing seabirds, their chicks and their eggs. Gull egg collection (for food) was a major activity for island populations who couldn't afford to be picky. Witness the annual muttonbird (shearwater) take still carried out in New Zealand today. St Kilda and other islands in the Hebrides were also known for eating gulls.
'Odd' seafood will also be a regular part of their diet - particularly in famine conditions. Seaweed, whelks, cockles, seaslugs etc. Whatever can be scraped off a rock at low tide will go in their stew pots.
Depending on the climate of your peninsula, they may also have seasonal gluts of insects to eat: grubs, crickets, locusts, wasp larvae, bee honey comb (complete with bee larvae) and so on.
Agriculture - whether farms or gardens - will rely heavily on human and animal waste as fertiliser. Given the size of the peninsula, I doubt that there will be many large domestic animals around. They need too much pasture.
They cannot afford to be picky, especially when food gets scarce.
On the one hand, this means they will likely eat carrion, sick or wounded creatures ("I can eat that staggering possum now, and get Tuberculosis in 10 years time, or I can not eat it now and be dead next week from hunger"), and plant food that's entering the 'slimy yet satisfying' stage: wormy apples have extra protein after all... This all carries some risk in terms of health, but as I implied earlier your people will have to choose between eating something rotten or starvation.
On the other hand, this may also lead to an interesting food culture in terms of fermented things. They have access to salt (sea water) which can be boiled to brine to make a preservative. Fish, shellfish and vegetables can all be preserved in salt brine. Other fermentation methods do not use salt (eg Kaanga pirau: rotten corn). Fermentation has the benefit of trapping valuable vitamins in some food, as well as preserving the food long term, which is vital when the quantity of food is not guaranteed.
I mentioned gluts of insects above. However, they will also be absent as a or hard to find at some times of year, especially in a temperate climate.
This goes for the majority of your food sources. Seabirds will likely nest only at certain times of year, plants grow slowly over winter if at all, and adverse weather conditions may prevent fishing during the equinox, to give a few examples.
Your people, being cut off from the mainland and its food sources, cannot import food to balance out those cycles and will regularly face certain times of year when they have low food resources available. Late spring is a possibility. You have used up your winter stocks, crops are growing but haven't matured yet, the weather is too unpredictable to fish and so on.
Then you have to think about famine.
Sometimes, perhaps in those hungry times of year, your food resources will not renew themselves. Maybe your people anticipate seabirds to arrive and nest in early summer? Well this year they will not nest well at all, and the fishing is very poor too. Here, the local climate of the ocean and its currents has driven your food away from shore - as the currents move, the fish move away. As the water warms too much, the shellfish die.
If this situation is combined with drought or flooding that removes another food source temporarily you have a serious problem on your hands. Famine will kill your young children and your elderly through starvation or diarrhea or any number of famine related causes.
How they cope with starvation years will influence their culture in the long run, and will also serve to manage your population numbers, potentially allowing other food resources to bounce back in later years. Quick breeding animals like seabirds will recover their population levels faster than humans as a general rule as environmental conditions return to normal.
tl;dr: There will be years that your population crashes, but through a combination of eating whatever they can get, and not being fussy about the state that it's in, they should be ok in the mid-to-longer term