The primary advantage of a propeller is that it is more efficient than a jet engine. Moving large masses of air slowly is simply more efficient than moving small masses of air quickly. In other applications we see helicopters are far more fuel efficient in hover than a Harrier "jump jet", or propeller driven ships more efficient than pump jets or other forms of water jet propulsion.
In fact, you might even wonder why use a jet engine at all, given these factors, but most users who have other considerations like speed, the need for a very compact powerplant, silence in operation (pump jets) and so on find these considerations have far greater priority than fuel efficiency.
GE 36 Unducted Fan prototype
Unfortunatly I can no longer find the link outlining performance, but back in the 1990's there was experimentation with something called an "unducted fan", which was a form of turboprop which dispensed with the usual gearbox and used the turbine to drive the unducted fan blades with minimal speed reduction. While this was extremely noisy (the main reason that it was never adopted), there were great gains in fuel efficiency, and the large area that the unducted fan could "grab" when running gave the plane far superior performance in things like short field take off. The Soviet Union and now Russia have done more development work on unducted fan engines (they have far less stringent requirements for noise reduction, and most of their engines seem to be used on military transports)
Antonov AN-70 prototype
Progress D-27 Propfan engine
So the primary advantages of propellers is their efficiency (within the correct flight regime - no propeller can power a supersonic aircraft, for example) and greater performance in certain aspects, like short field take off. The power plant is actually irrelevant, model aircraft are powered by rubber bands, two stroke "glow plug" engines and electric motors, while most common propeller driven aircraft are powered by piston or turbine engines. A steam engine for propeller aircraft was developed in the 1930's, the Germans used diesel engines for a period during that time as well, and any other motive power source, like a Stirling engine could be used as well.