By a stable orbit I mean not getting ejected by the other planets or ejecting them, Let's create a scenario where Neptune occupies an orbit in the asteroid belt at 2.76 AU, Ceres average distance from The Sun. Since the distance of the asteroid belt varies from 1.78 to 4.2 AU, I decided to use the orbit of Ceres 2.76 AU for the placement of Neptune. If this swap happened immediately how fast would it take for Neptune to clear an orbit? And how bright would Neptune appear to the naked eye from Earth?
I don't have the simulation resources to put numbers on it, but several things would happen.
Firstly, Neptune starts to warn up considerably, because it's much closer to the sun now, and will shed some atmosphere in consequence of this. That probably won't have a whole lot of effect.
Secondly the orbits of everything in the solar system start to change. You've removed Neptune's influence from its old position, and plonked it down near Jupiter. Its interaction with Jupiter is the most important thing. Their orbital periods start at a ratio of 1:2.58, but will change. In this process, most of the asteroid belt will be scattered all over the place; Neptune will acquire some Trojans quite quickly (Ceres has temporary ones). Neptune and Jupiter will have impacts from plenty of asteroids, and everything solid of any size is at risk.
For timescales, this will take a few hundred orbits to progress far enough to get an idea of how it turns out, so a thousand years plus, and 10-100 times longer for everything to stabilise, at which point the neighbourhood would have been "cleared". The exact sequence of events will depend on where Neptune appears in Ceres' orbit and what asteroids are nearby. I don't know if Neptune will end up in a stable orbit, get ejected, or collide with Jupiter, but those seem to be the likely options, and the collision would be extremely spectacular, until the debris arrived.
Overall, doing this looks like a fine way to mess up the solar system. If you're thinking in terms of Larry Niven's A World Out Of Time, where Earth is moved via moving Uranus, please remember that book was written when we had a rather limited model of the Solar System's dynamics, and well before we started to understand ideas like planetary migration and ejection.
Interaction with Jupiter would be very strong (that's why no planet formed in the asteroid belt, too much interference). A first few million years, orbits of all gas giants would shift, probably outwards, to reach a more stable orbit (Neptune would become the new Jupiter). The solar system actually started out more closely packed, and drifted apart until the mutual gravity no longer had such a strong effect (https://en.wikipedia.org/wiki/Nice_model). The inner solar system would first suffer another heavy bombardment due to scattering of all the asteroids into unstable elliptical orbits. The disturbances because of another heavy planet would also likely shift their orbits a lot, possibly sending them on a collision course (but very unlikely). After a few hundred million years, things would stabilise again.
More generally speaking, any modification of a solar system would have quite drastic consequences. Many body problem is a very chaotic physical system, and most initial conditions that are not carefully thought-out, will be less stable than the current state, and will only find a more stable state over time.
As for the colour and brightness: at the distance of Jupiter, Neptune would have about 10% brightness of Jupiter (based on difference in albedo and radius). At the asteroid belt position, which is about half the distance of Jupiter to the sun, it would thus receive 4× as much sunlight, and also be 4× as big in the sky, overall being about 1.6× current brightness of Jupiter. Of course, heating up would change its albedo (probably becoming brighter due to different coloured clouds in the outer atmosphere).