Yes. The configurations you describe are entirely possible and are believed to have direct analogues in the Earth's past. We currently have an East-West divide in our major continental bodies (Americas vs. Afro-Eurasia), and the North-South split is pretty much what the Tethys ocean did to godwanaland about 200 million years ago if I recall correctly.
If you wish to get technical, plate tectonics and RRR-triple-junctions would give you the mechanism you require for splitting a supercontinent into two (google the formation of the atlantic ocean for less technical references), and the final trench closure prior to the formation of pangaea-esque supercontinents also provides a similar geography.
by ocean i mean not a sprawling one like earth has but more like a
strip spanning either along the equator or along the meridian.
How much "mostly land" are we talking about then? Your continents are fully separated by water right? If you wish to maintain a vaguely earth-like biome you'll need large oceans to act as a reservoir for rainwater. Even then, the interior of supercontinents is INCREDIBLY dry due to water sources being a long way away (rainclouds exhaust water prior to reaching the central interior). If the planet you're thinking of is 75%-80% land it will be very very very dry and desertlike.
Regardless, the plate tectonics I outline above are unchanged (barring some technical details about dewatering reactions' importance in lowering mantle viscosity near trenches). You can totally have the scenario you describe. If the oceans are very small you might have exposed basalt (Earth's oceanic plates) between the continents -- which seem to rise above the basalt plains like high mesas.