Storms are driven by energy. So you need to consider how and where energy enters your fantasy waters.
Some main rules of thumb:
- The larger the expanse of water, the more wind energy can interact, driving small ripples into large waves and adding energy to water.
- Ocean waves might appear shallow but the oceans are deep so quite a bit of the energy enters the water column - the vertical space that waves and shallower currents occur in. When that water column hits shallower water or a conflicting current, the energy it contains may lead to wilder, larger or less predictable waves.
- Cyclonic and anticyclonic storms (cyclones, hurricanes) are driven by heat and evaporation, the warmer the water the more intense the storm. They die after a while over land when they are cut off from their ocean power source.
- Areas where powerful wind/current systems tend to meet, will have more variable weather. For example, the UK, sitting on the gulf stream and the jet stream, between warm Atlantic/Europe and Scandinavia/Russia/arctic, and between a large body of water and a large land mass, has variable unpredictable weather, compared to many places.
This is why storms build up around the southern ocean (deep, no land mass blocking wind or water flow), and across the Atlantic and Pacific, but not really across the Arctic ocean (broken up by land). Its also why the storms we associate with the tropics, forming across the Atlantic/Pacific, tend to be cyclonic (heat driven hurricanes/cyclones) while those in the southern ocean are more often very high winds and waves without a formal cyclonic/anticyclonic structure.
Bad waters aren't always storm driven. Expanding on the question, intense waves and water currents can exist where currents conflict, or are forced into a narrow area - especially a deep and narrow area with room for large water movements. So also consider where waters meet.
This is a large part of why the Cape of Good Hope (southern Africa tip) and Cape Horn (southern Chile tip) were both so ill-reputed to sail - conflicting currents between the oceans that meet there, as well as the winds around there. Both built up over thousands of miles of ocean surface, too. Its also the reason for tumultuous passages like the Maelstrom in Scandinavia.
Last, consider land topology such as mountains. These can block wind and weather systems, or funnel them - sometimes causing conflicting wind systems. They also tend to create a rain shadow, and may give rise to a desert on the far side.
Finally, not predictable at all, but rogue waves can appear anywhere, even far out at sea, from random wave energy behaviour. They can be more likely in some areas, but are essentially randomly formed, not storm created.