The thing about gravity is that in reality it should always pull.
There are four different types of forces in our universe:
Electromagnetism - The force controlling electrical currents (think electricity in a wire).
Weak Interaction (or Weak Nuclear Force) - The mechanism of interaction between subatomic particles that causes the radioactive decay of atoms (think hydrogen and helium in the sun).
Strong Interaction (or Strong Nuclear Force) - Is similar to weak interaction but just much stronger (as the name implies). It decays particles much faster.
Gravity - Is the idea that objects in the universe are attracted to each other because space-time is bent and curved.
The reason why gravity pulls everything down or towards the centre of mass is because of Einstein's theory of general relativity. According to the theory, everything weighs down on spacetime and bends it depending on anythings weight. The heavier something is the more space time bends.
You can imagine this quite easily if you think about a trampoline and a ball. If you stand on a trampoline, it bends down where you are standing and the ball would roll towards you. This is essentially how general relativity works, in a 2D example anyway. Across the universe, only examples of gravity pulling can be found but in theory you can imagine gravity pushing as someone under the trampoline pushing you up (this would likely require negative mass, which sadly explodes on contact with matter so not really an option), but since that is (as far as we know) beyond our laws of physics, I will discuss a different option.
A possibility for your liftstone could be using magnetic levitation (electromagnetism) or maglev for short. Since our gravity cannot push, you would have to use a different force that the floatstone could emit to overcome gravity. If the float stones had magnetic properties strong enough they push away from the earth itself, holding your ship in the sky. But as wikipedia puts it "The two primary issues involved in magnetic levitation are lifting forces: providing an upward force sufficient to counteract gravity, and stability: ensuring that the system does not spontaneously slide or flip into a configuration where the lift is neutralized."
The first one should be simple enough if you are willing to step into the realm of fiction (with flying ships you seem to be willing) and just say that the floatstones produce such an upward force through magical means, or at least it isn't understood how it works in the era your world is in. If you are wanting to stay away from just saying "it's magic!" then just saying it is not known why it works in your world is the better option.
The second problem is just a matter of design and where you place the stones in the ship. You have to have the magnetics evenly distributed in the ships design so that the ship doesn't tip and rock about. One way to do this would be to have the stones higher so that the ship simply hangs down of them. But if you wanted to maintain the 14th century ship design maybe have it in the crows nest on the mast. Having one floatstone could just make it rock if the ship was unbalanced or had people moving around on it so more than one would probably be a necessity. You could have them in multiple masts or in some other configuration, but I recommend have them above the main deck so the ship won't flip upside down.
Medieval ships ranged about 40 ton to 120 ton and so depending on the size of your ship you could need a different amount of these stones, but this is not a number I can calculate because of the unknown magnetic strength of each stone, but more stones the merrier I guess. Assuming they have to be built into the ship like any kind of material, they ships maybe need a light but strong metal to reinforce the hulls of the ship which can then hold the stone in place.
Lastly I suggest that you would want this floatstone material to repel when it is hotter rather than colder (ship rises when hotter I mean) for 2 reasons. If something is hot it has more energy which would make sense for the floatstone because the more energy it has the more it could oppose gravity. Secondly, at higher altitudes it gets colder and there is less oxygen in the air. This means when it rises the floatstone will cool down because it is surrounded by cold air and so it would stop rising and reach a point where it is in equilibrium and the ship is stable. If it rose when it got colder, it would keep rising and rising faster the further it got from the Earth unless you could keep it a constant temperature with no fluctuations, but this is risky because if the blacksmith gets tired or fails to keep it at the right temperature it will go out of control very quickly. Adding to this point the lack of oxygen at high temperatures means it is harder for fire to burn so when you rise the fire will burn less, and if you make the floatstone sink with less fire, it will regulate itself to some degree. For this to work, the floatstone would have to be extremely sensitive to heat.
I should say I'm not sure what consequences would occur creating such a strong magnetic field in such a small area. Just pray it doesn't attract lightning.
I hope that answers some of your issues. If you have any questions let me know and I'll edit this response.