Okay, I'll throw my two cents into answering this question after thinking about it for a long while.
A single coral planet? Probably not.
When most people talk about "coral", they're actually referring to a collection of organisms, not a single individual coral polyp. You can see these individual polyps with a magnifying glass or good macro lens:
Photo: Mary Lou Frost. Courtesy of Coral Reef Alliance.
These coral polyps actually look a lot like sea anemones, and are usually only seen during the nighttime when they come out of their houses to feed on drifting organic matter and small plankton in the water column. Contrary to popular understanding, corals are truly animals, not plants, and they do need to consume organic matter to survive. They're famous for their symbiotic relationships with zooxanthellae which can provide them with simple sugars produced by photosynthesis, but nutrients and more complex molecules are obtained by heterotrophy of passing organisms.
"Corals", the large-scale structures we're used to, are more accurately referred to as coral heads. These form because many of these organisms work together to secrete calcium carbonate skeletons. The structure left behind is composed almost entirely of crystalline calcium carbonate, better known as calcite.
This is a problem for a coral-based planet because a single coral doesn't produce the kind of ridges and valleys you're looking for - it only secretes calcium carbonate behind it, leaving a kind of "cup" shape in which the coral lives.
Okay, a "coral head" planet then - how about that?
If you're okay with a planet made up of many coral polyps, as it seems like you are based on your provided image, things get a little more reasonable. There's a fuzzy line here, however, between a planet made up of many coral polyps and a planet that's simply covered in coral.
There are two main challenges I see in a planet-sized coral head. The first is to do with formation, and the second is to do with maintenance and stability.
Coral head planet formation:
What's required here is water. Coral polyps, at least as far as current biology goes, require water to survive. Removing a coral from the ocean for even a short period of time is usually fatal for the coral because it has no way of preventing water loss. Water keeps the coral polyp upright and sprightly because corals have a hydrostatic skeleton and rely on water pressure to keep them sturdy. Additionally, I don't know exactly how corals secrete calcium carbonate, but if it's in an ionic form rather than calcite nanospheres or something, then the water will be required to grow the calcite at all.
The bigger problem here is mass conservation. Coral heads form after years and years of steady growth, in which mass is taken from the water around them and added to the coral skeleton itself. For a space coral to grow in the same way, it must obtain mass from somewhere. To stay true to modern coral science, it can't be just any mass either - it needs to be calcium, carbon, and oxygen (and probably some hydrogen). These elements, while quite common in most planets, aren't very volatile and are relatively uncommon in interstellar space. To have a coral "grow" and end up in the shape you're hoping for, you'd have to explain where those molecules are coming from.
Finally, there's a proportionality between the size of the coral house (and thus the ridges formed) and the size of the coral itself. Brain coral polyps, as you use in the image above, are normally a millimeter or so in size, and they produce topography on the scale of centimeters. To get planet-scale topography with current biology, we'd expect moon-sized organisms.
Stability
If you've got solutions to the above problems, awesome! Let's push on to talking about the stability of the recently-formed planet-sized coral head. What it seems like you're interested in here is the deep valleys and high ridges of a brain coral. This planet-sized coral, if not covered in water, will definitely be dead. The coral polyps will have long since rotted away, leaving behind their calcite skeletons. The fundamental constraint here on the heights of these ridges and valleys is therefore the compressive strength of calcite.
Caution: what follows is a very vague estimate with lots of spherical cow math:
Coral biologists, the meticulous people that they are, have of course measured the compressive strength of corals (paywalled journal article, sorry, but the important number is in the available abstract). They record the compressive strength as between 12 and 81 MPa (megaPascal), which is "lower than most other carbonate skeletal materials, but higher than that of carbonate engineering materials like concrete and limestone". I assume that the "other carbonate skeletal materials" include things like human bones, but I was impressed to learn that coral skeletons are apparently stronger than concrete. We can check the claim made here against third-party estimates of limestone and concrete and confirm that, in fact, coral's upper limit of 81 MPa is indeed higher than that of limestone (60 MPa) or "Portland Concrete, 28 days old" (35 MPa). Fortunately, this site also includes the compressive strength of granite, which allows us to do large-scale stability comparisons of a theoretical coral planet to our own, mainly granitic one - allowing us to avoid some very scary maths necessary to calculate it by hand.
Granite has a compressive strength of 130 megapascals while coral maxes out at 81. Thus, naively assuming similar properties as a first-order approximation, we'd expect coral-based landforms to be about 62% as extreme as the ones we find on our own planet. With a particular famous mountain clocking in at 8,848m and a less-famous canyon measuring 5,800m deep at its deepest point, we can estimate that a coral-skeleton could cause a landscape with nearly 12 kilometers of relief. Sadly, this isn't nearly as much relief as we're hoping for. The image of the brain coral you use as an example of the topography you're looking for has a valley-peak height approximately 5% of the coral head diameter. With an Earth-sized planet, our 12km relief is barely 0.1% (12km / 12,000km). Darn.
Note: I'm using Yarlung Tsangpo rather than the Marianas Trench for two reasons. One, the Marianas Trench is actively forming and not a stable system. Admittedly, Yarlung Tsangpo is also still being eroded, but not at the same rate. Two, the Marianas Trench can only get as deep as it does because it's braced by the water in it, and we're hoping for a water-free world here.
However, don't give up hope just yet! Part of the joy of Worldbuilding is being able to tweak all kinds of constants. We can improve on our previous estimate by reducing the mass, and therefore the gravity, of our planet. Rather than using Earth as a model, let's look at a place like Mars. Mars is famous for having the largest mountain of any planet, nearly 22km tall by some estimates, and has some pretty darn big valleys as well, dropping to 8km below the "sea level". Mars' diameter is about half of Earth's, and the relief is nearly triple Earth's, so we've improved from 0.1% to maybe 0.5%, or one-tenth the relief we're hoping for. By pushing the mass of the planet even lower, we can get higher and higher relief and a lower radius. Mars' mass is about 10% of Earth's, so some back-of-the-envelope math suggests that reducing your planet to a mass about 0.5% of Earth's will produce the topography you're looking for. That puts your planet at about the same size as Pluto! (Whether this is still a planet or now a "dwarf" planet is not a debate I am interested in right now).
Whew. Of course, I handwaved a lot in there, and reality would probably be even less favorable. Some additional factors that should be considered in a more rigorous calculation: the angle of repose for calcite (because the high and low points that I use aren't right next to each other, and the slope between the two should be considered); the compression of coral and its porosity with depth, as it metamorphoses into marble; the possibility of spinning the planet faster to mitigate some of the gravitational force; and the addition of a thicker atmosphere to help support the relief.
Theoretical Worldbuilding setup
Can I stop unbuilding now? Yay! How I'd imagine a coral-based world coming to be would be something like the following.
A ball of water about the size of the Earth, supersaturated with carbonate ion forms, orbits a warm sun, keeping the planet tropical and friendly. At some point, a space coral spore finds it and begins living in the water. Over time, this coral reproduces and proliferates, and continues to grow in both size and number. As it grows, the calcite skeletons are jettisoned to the center of the planet, where calcite begins to build up and conglomerate. Over millenia, the corals cover the surface of the planet and all the calcite in the water is slowly used up. Over this same time period, the water is gradually lost to space , facilitated by the warm sun and the nearly-100% H2O atmosphere. Eventually, a crisis point is reached as the corals perish, leaving behind their beautiful homes and instead converting their remaining biomass into other space coral spores to be distributed on the solar wind. The planet is slowly freeze-dried as the sun continues to evaporate the last bits of water, minimizing water-mediated erosion of the surface. The resulting planet is about the size of Pluto composed of nearly pure calcite, with a topography dramatic enough to be seen from space.