It's an interesting question... Personally, I don't think it would impact/change us that much, because we in many ways have put ourselves "outside" the grasp of evolution.
From nature, we're not that well endowed -- neither with warm fur, protective scales, long claws or long teeth. We're not especially fast and we're not very good at hunting "by hand".
What separates us from other animals are: Our opposable thumbs, which make it possible to manipulate objects. Together with a big and creative brain, this allows us to make tools. Language -- including the use of oral and/or written -- allows us to share our discoveries to others and across generations.
So a group of humans stranded on some deserted island, would do as every other humans in a fix -- use our big brains to plan and invent tools, make tools with our hands, and then share it among our population. So it would be our tools -- weapons, shelter, communication, fire, cloths -- that would get us out of the fix, and would allow us to start using new sources of food and such when needed... not evolutionary changes to our body through the generations. Therefore, I think our bodies would remain virtually unchanged -- for thinking and hand-manipulation, it's already as good as it gets!
What separates us -- humans today -- from the humans many thousands of years ago, is not our body or brain; but our knowledge -- and the effectiveness of which we can communicate this knowledge between ourself and to the next generation. We literaly stand on the shoulders of all the great (and less great) thinkers that came before us, and can use their discoveries and mistakes as a base for our own thought. That is what has given us todays technology, and it's that combination -- generations of discovery coupled with technology -- which truly separate us from humans from 10 000 or 20 0000 years ago... not evolutionary changes.
That said, when humans -- or a group of humans -- comes under pressure from things like famine or illness; sure we'll see evolution playing its hand, separating out those most suited to live: The Black Plauge and Smallpocks... HIV/AIDS in Africa (there are some with natural resistance, they stand a better chance to pass along their genes where there is little health-care)... Scikle-cell animia and malaria... The way illnesses that are deadly when first encountered, becomes "childhood illness" which we just live beside...
But all in all, humans usually overcome our problems with knowledge, planning and tools; and thus put evolution into the back seat. Trapped on an island, humans would conquer various habitats and food-sources by inventing tools -- and then pass this knowledge to future generations. No need for evolution to adapt our body then -- and of course no opportunity.
A stranded population would undoubtedly develop its own culture and language -- and probably their own religion and laws. But this wouldn't be passed by genes. We may of course get some class-separation -- where the lower class toils for an upper class -- and where of course some classes would have a better chance of surviving than others, but that is "survival of the richest" -- not necessarily the fittest as evolution defines it.