A valley created by tectonic plates "like a wound on the continent" describes the Great Rift Valley in Africa. It is very much being made by tectonic plates.
The Wikipedia article walks through how rifts like this eventually fill up with water and so a swamp would be halfway. The real Rift Valley is immense and has all kinds of things in it including other mountains, and of course like anything so big the highlands along the edge are heterogeneous and not all giant cliffs.
You could hybridize your rift with something like the Grand Canyon to get steeper walls. The Colorado has been working on the Grand Canyon for a long time but it is possible for a single catastrophic glacial melt event to carve deep channels in a very very short period of time. The Hudson is one of these. From https://pubs.er.usgs.gov/publication/70030018
The valley and its distal deposits record a discrete flood event that
carved 15-m high banks, formed a 120-km2 field of 3- to 6-m high
bedforms, and deposited a subaqueous delta on the outer shelf. The HSV
is inferred to have been carved initially by precipitation and
meltwater runoff during the advance of the Laurentide Ice Sheet, and
later by the drainage of early proglacial lakes through stable
spillways. A flood resulting from the failure of the terminal moraine
dam at the Narrows between Staten Island and Long Island, New York,
allowed glacial lakes in the Hudson and Ontario basins to drain across
the continental shelf. Water level changes in the Hudson River basin
associated with the catastrophic drainage of glacial lakes Iroquois,
Vermont, and Albany around 11,450 14C year BP (∼ 13,350 cal BP) may
have precipitated dam failure at the Narrows... Based on bedform
characteristics and fluvial morphology in the HSV, the maximum freshwater
flux during the flood event is estimated to be ∼ 0.46 Sv for a duration of ∼ >80 days.
So: a rift like the Great African rift, and then a glacial melt event coursing down it to further carve out your steep cliffs that will help keep the dinosaurs down in there.