On Earth, sailors would navigate the oceans at night by using the stars as guides; since there are no landmarks in the ocean like there are on land (save for the occasional island), that's pretty much all they have. In my world, however, there are no stars; there is only the world, the sun, and the other planets in the solar system. The rest of the universe is empty. Without stars, how would travelers, and especially sailors, navigate and tell where they're going?
First off, this question reminds me of the Doctor Who episode The Pandorica Opens, in which the TARDIS exploding destroys all the stars.
To get on with the main question, there are four ways that ancient peoples generally navigated.
First off, your hypothetical people could make like Boy Scouts and use a compass
Seriously, this is one of your best options. Not only are compasses easy to make, but they are also pretty reliable. While you do have to deal with the whole magnetic N pole vs. actual N pole problem, the great distances involved in sea travel make this a moot point. As a former Boy Scout this is my favorite option, as I can vouch for the reliability from personal experience.
A second method is gyroscopic guidance systems. While the machinery involved is *somewhat* complicated, it is still doable with ancient technology levels. That being said, this approach does have the major downside: gyroscopes tend to break. A lot. As a result, captains would probably keep gyroscopes as a secondary guidance system.
The Gegenschein effect. This would definitely work, but it would be somewhat hard to do. Even though the lack of stars would make the Gegenschein much more visible, it would still be hard to navigate by. Also, most people have not heard of this, so readers would unfortunately consider it a particularly bad attempt at handwaving.
Finally, they could just not navigate. No, seriously, historically most captains did not navigate at night. They just did not think it was necessary, as the large distances involved in sea travel meant that they could generally afford to let their navigators take the night off.
Well... I guess they would have to go by the other planets then, the Gegenschein could also help:
Gegenschein (German: [ˈɡeːɡənʃaɪn]; lit. "countershine") is a faintly bright spot in the night sky centered at the antisolar point. The backscatter of sunlight by interplanetary dust causes this optical phenomenon.
Since you don't mention the tech level they could use anything ranging from a compass to GPS.
No explanation could be better than real history. Do what the Polynesian sailors used to do. Apart from relying on the sun they were able to read the sea itself to know their position. They were aware of how the water behaves at certain locations on a particular period of the year. They used things like swells, currents, wind directions, even a slight change in temperature of the water as their guide.
They even had a Navigational device, made out of sticks to depict the map, where each knot in the stick was either an island or some kind of intersection of two types of water. Follow the link for better description.
I saw a youtube video on it once, I can't seem to find that particular video, but various others are there, they should be able to help you.
Following coastlines is a great way to not get lost, for people without advanced technology. And as mentioned in another answer, you can get a bearing with a compass.
Ancient mathematics would be enough to navigate by the planets. Most planets are more or less in the ecliptic plane - to ancient astronomers, the sun and all the planets they could see moved across one single line in the sky. You could calculate your position relative to that line. Observing where on the horizon the sun and planets rise and set allows you to figure your bearing and latitude.
With a sextant, a good clock, and an astronomical almanac, you could calculate your bearing, latitude and longitude from the planets or the sun. The invention of pendulum clocks was a huge help to accurate long distance navigation in the real world; before that, your longitude was a bit of a guess until you saw land.
Nobody else seems to have mentioned it, so I will - the three moons of where-the-heck-am-I make for a very fine navigation system.
OK. Yes. You did say just the local star and other planets, but if you are wanting a navigation system, then throwing in a few moons may help.
Even if this solar system is all that exists in your universe, it can still be all they need to navigate. Some of the brightest 'stars' in our sky are simply planets in our solar system (like Mars and Venus), so as long as your solar system has a few of those, they can be used for navigation.
If they absolutely need to...
Well, as already mentioned, usually sailors keep close to the shore, especially in ancient times. Yes, there was this "Columbus" guy who wanted to sail to India via the western route and almost perished had not America been in his way, but in general no one was that stupid.
In real life, Germany has still three firevessel in active service; they are unmanned and carry enough Diesel fuel for more than 400 days of operation. They are also designed to serve shipwreck victims (accommodation, an emergency radio and a helicopter platform to get them off the ship). Sorry, no english Wikipedia article, just the german one: https://de.wikipedia.org/wiki/Unbemanntes_Feuerschiff
Obviously, there are limits to where you can station lightvessels. As a rule of thumb, assume 6 meters of anchor chain of every meter of water depth. I assume that for very deep water, you need to have buoyant bodies at regular intervals of your anchor chain, to compensate for the weight.
On Earth, I guess that a chain of lightvessels between America and Europe would not be have been commercially viable. However, for a hypothetical not-too-distant island or continent, this might be done.
Using accurate clocks
In order to know where you are at sea, you need to know both latitude and longitude.
Latitude is some sort of 'trivial' problem since ancient times, because you can tell at where latitude you are by measuring the duration of the day and night.
Now, longitude is a more difficult problem: it was a problem even in the earth, because at sea you cannot always see the starts, due to clouds.
On top of that, to calculate your current longitude by looking at the night sky is not trivial.
However, an accurate clock can, with some training and preparation, tell you at were longitude you are by comparing the sunset time against a known location. This was, in fact, the famous solution proposed by John Harrison, which actually won the longitude prize.
Accurate is important, because until Harrison's H4, mechanical clocks in ships used to be completely unreliable, due to the severe conditions, temperature variations and the ship's movement.
Is it "without stars"? Or is it "without nearby, resolvable stars" (say, a star ejected from its home galaxy together with the habitable planet, but given enough time and darkness, the galaxy still visible)? You will get a short interval between the daylight and the total darkness when eyes adapt and the sky stops being uniform. Maybe only blue-eyed people able to see the galaxy?
1.The Sun is a star, so "having no stars" is a funny paradox.
Well I'm no sailor, but is there a moon in your world?Because if so maybe it could be used as a means of direction.Or at least as an "anchor" of sorts so those on the seas know where to look for for reference.
A compass might work, but how about also using some knowledge of water currents' paths and temperature?
You might consider a system embedded in a nebula as a way of eliminating the view of stars rather than that star being the only one in the universe. Douglas Adams did this with his Krikkitmen.
If compasses are too mundane and you want an exotic/steampunk solution you might consider a balloon-based lighthouse. If the lighthouse sent a beam of light straight up and illuminated a reflective balloon, that balloon could then be seen for miles.
The main drawback, the line of sight would be limited by the curvature of the surface of the planet.
One possible method of navigation would depend on the features of the solar system the story it is set in.
If another planet in the solar system happens to be visible in the sky at night its angle above the horizon and the date and time can be used to calculate the latitude the ship is at. That is similar to using stars to find the latitude, but more complicated since the date and time will be necessary.
If your society has telescopes they might be used to find the time by studying other worlds and comparing that time with the local time found by the time since local noon or sunset.
The four Galilean moons of Jupiter were discovered by Galileo in 1610. Their orbital periods around Jupiter are 1.769 days (Io), 3.551 days (Europa), 7.155 days (Ganymede), and 16.69 days (Callisto). So every time that they were observed their relative positions would be at least slightly different.
So if a table is compiled showing their predicted positions at regular intervals at a location A, with the local time included, and if they are observed from point B, and the date and time of such a configuration is found, the difference between the local time listed for point A and the local time found at point B will show the longitude difference between point A and point B.
I believe that this method has actually been used on land to accurately measure the longitudes of various places and map them.
It is somewhat harder to use this method at sea, so marine chronometers turned out to be the successful method for finding longitude at sea. But possibly a society on an alien planet might find a way to use a similar technique at sea.
The first atomic submarines were built about 1960.
The main advantage of atomic submarines is that they can and often do stay hidden underwater for days, weeks and months at a time, while diesel submarines have to surface for hours about once a day to charge their batteries.
So atomic submarines travel underwater for months at a time while unable to see the Sun, the Moon, the planets, and the stars and unable to navigate using them.
So the methods of navigation used by atomic submarines while under water would be interesting. Some methods might require high tech equipment and others might use more low tech equipment.
At depths below periscope depth submarines determine their position using:
Dead reckoning course information obtained from the ship's gyrocompass, measured speed and estimates of local ocean currents, this could also be considered an estimated position as long as the ocean current is computed in.
Inertial navigation system is an estimated position source, utilizing acceleration, deceleration, and pitch and roll for computing.
Bottom contour navigation may be used in areas where detailed hydrographic data has been charted and there is adequate variation in sea floor topography. Fathometer depth measurements are compared to charted depth patterns.