A planet-sized cube will collapse under its own weight into a sphere. The comet you mentioned has too little mass for that (and it would become a sphere if it was liquid). Our Earth is essentially a bubble of lava, with a thin solid crust. And it is the weight of the rock that turns lower levels into liquid lava.
So I will assume your cube planet is made of unrealistically strong materials that prevent a collapse (or it is a hollow artificial construct), the vertices of the cube will work like mountains and edges -- like mountain ridges. So you cannot sail from one face to another. Instead of planet-wide ocean, you will have lakes in the middle of each face. See Figure 1 here: https://arxiv.org/pdf/1206.3857.pdf
In fact, you might not be able to walk or fly across the edge, as earth-sized atmosphere will be nonexistent near the edges. Gas (just like a liquid) will try to form a sphere around the cube, and if there is not enough of it, the edges and vertices will stick out. In fact, you will have a ring of habitable land on each face, between the central lake, and the end of the atmosphere. In fact, you can have completely different atmospheres on each face.
If you have enough gas to encase the entire cube (see Figure 3), the atmospheric pressure will be a lot higher, giving you (I think) a Venus-like hot world
People will notice the change in gravity angle as they travel towards edges (Figure 2), but they will just think that they are climbing a mountain ridge. Fog and dust in the atmosphere will prevent people from seeing all the way across the face of the cube, so they will assume they live in a square-shaped valley surrounded by impassable mountains.
The first proof of Earth shape was The Eratosthenes Experiment (250BC):
It was based on the angle of the sun. On your cube, the sun will have the same angle with the surface. But just as Eratosthenes, your people will measure angle vs. up-down gravitational field line (see Figure 2). So exact replication of his experiment (with 2 points, North/South of each other) will lead to the same conclusion: the planet is round.
Repeating experiment with points East/West of each other might show that shape is not round, or doing the experiment with more than 2 points. I am too tired to figure out the angles :)
user535733's idea of measuring time between sunrize and sunset is valid, but requires a clock (renaissance technology) or lots of patience with hour-glass and more complex theoretical argument.
The "magellan" approach of travelling across faces requires oxygen tanks and breathing masks, or space rockets, or tunneling under the ridge/edge.