The answer is no; they'll not even be able to withstand normal human acceleration limits. Not because of the pressure (induced by constant acceleration), but because of the momentum change in their environmental medium (caused by the sudden changes in acceleration, especially at the beginning of the launch).
A simple experiment that explains this (don't try this at home edit - for ethics reasons, not because I'm worried about the results); take a small fishbowl, and put a lizard in it, seal the top and shake it. Afterwards, the lizard will be sore and more than a little pissed off at you, but it will be alive. Take a fishbowl full of water and a goldfish, seal the top and shake it. Your fish will be dead, almost instantly.
Further Edit - Another analogy that you might consider here is the shockwaves caused by Blast Fishing, which also causes large changes in momentum over a short time.
Why? Because water is a non-compressible medium (and it's very dense). In any collision, the most deformable object is the one that also absorbs as much of the kinetic energy of the impact as it can. This is why modern cars are so 'flimsy' by comparison to older cars, and consequently so much safer. In an accident, the car breaks so that it absorbs as much of the kinetic energy as it can before transferring the balance to you. Older, more rigid cars don't do that and as a result many people found out what being the most deformable object in a collision actually meant right before they died.
Air is very compressible (read as deformable in this answer) meaning that it can absorb a lot of energy by comparison to its mass. Unfortunately, that doesn't mean much because its density is very low, so the two tend to balance each other out. Still, our lizard only has to worry about being the deformable object every time he hits a wall on the fishbowl. Strap him in, and he's going to fair a lot better (hence seatbelts).
Your fish on the other hand is in a medium that is not only non-compressible, but very dense. This means that it's going to take a LOT more energy to launch all that water because of the mass, and more importantly, the sudden acceleration change is going to throw all that non-compressible mass your way as soon as you start your acceleration, likely crushing you. This also means that any form of maneuvering in space or engine issues that cause significant vibrations or shaking are of concern because of rapid and large momentum changes other than the constant acceleration along your line of axis.
The problem with the reasoning in your question is that buoyancy is not the same thing as pressure. Buoyancy is relative density, whereas pressure is the force of a mass being applied against you (this is a simplification but functionally correct). Sure, the pressure deep in the ocean depths can be extreme, but it can also be introduced to your creatures slowly - they don't instantly go from 10m depth to 1000m depth, and it would kill them to do so. But, in a rocket, that's exactly what you're asking them to endure.
They'd be far better in some form of gel, that allows them to breathe but is lighter weight and can absorb most of the impact for them. Water is not the best medium to find oneself in when face with sudden acceleration.