Things that scare us tend to be things that recently (in evolutionary terms) we had reasons to be scared of.
The darkness may hide predators or unseen traps. The cliff edge might be unstable and cause us to fall. The tight space might cause us to get stuck or unable to breathe, etc. Individual people might have more or less reaction to them but there is a genuine reason to fear.
To answer this question then you need to look for things that would be scary for an air breathing mammal but not for an aquatic creature, or the other way around.
Some examples, scary for aquatic aliens but not for us:
Pressure would probably be comforting for them, the feeling of being out in the air with nothing to hold them down or support them would be both scary and unpleasant.
Light might scare them, if they are deep sea dwellers then light might signal predators.
Being surrounded might be an issue, again depending on their usual habitat. If they normally live in deep water then being trapped, even if it's just with land underneath them, might feel claustrophobic. They can only run away in 5 directions if someone attacks!
If they are used to moving in big shoals then being alone might be incredibly terrifying.
If they use sonar then sound distortion or loud noises that interfere with it might be as disorienting and scary as darkness can be for us.
Some examples, scary for us but not for aquatic aliens:
Darkness is common underwater, but we really don't like it.
The feeling of pressure, all that water weighing down on us.
Aliens used to isolation might be surprised by our need for company. Or ones that move in massive shoals might be surprised by our need for privacy.
Heights, the aliens would have no reason at all to be scared of heights. When above water for whatever reason they may intellectually understand that falls are bad but they would have no "gut reaction" instinct telling them so.
Basically you need to identify something that is fearsome for one party and not for the other by looking at the difference in the environment. What is unknown or dangerous is scary.