I would like to take as a starting point the contribution of Anders Sandberg.
The basic assumption there is that the "mountain" is a solid (or near solid) and homogeneous piece of rock. "Near solid" still allows for fissures and cave systems but compared to the total volume of the mountain and the density of the rock, the reduction in overall weight they imply should be very small.
When considering beams, in mechanics, a well known result states that a hollow tube of equal mass to a solid tube (a rod) would resist bending considerably more than its solid counterpart. Other profiles with different "hollowness" are suitable too.
Therefore, without violation of the reasonable assumption that "a pile of solid rock can be so high as to not crumble under its own weight", it could well be that a structure can stand much taller if, for some reason it made "clever use" of its mass.
Yes, it will still break but with a lighter structural pattern, the "accumulation of height" is faster than the accumulation of mass and so it can stand taller before it hits that structural limit imposed by physics.
Examples where something similar to this is found in nature are certain trees, such as the Sequoia and the Baobab. Such trees stand can grow very tall and have hollow interiors. Their trunk does not grow as a solid "tube". This is beneficial for two reasons, it's not only that they are lighter (compared to a typical "solid" tree) but they can also resist bending, because of wind forces (for example), which, as the tree grows taller and taller and inevitably wider too, becomes a considerable force.
Therefore, if you relax the specification that the "mountain" is a solid, homogeneous, "rocky" kind of mountain, then you could end up with a much taller "mountain", in an earth-like planet with possibly an even more interesting (or flexible) narrative.
The key problem here now is how do you grow such a mountain that seems to be taking these principles into account?
There is an awful, obvious and quite boring option here, there are living things in the "mountain" and they build it and they interact with the story in mysterious ways until we discover that something is in the mountain and this and that the other.
A less boring option is that the "mountain" is one huge composite society of trees, with hollowed trunks, intertwined that grow and expand their base very very slowly. Existing processes accumulate dirt on its sides (by wind for example) and where there is dirt, water and air-born seeds (or carried by birds) there is the potential to form trees, etc. So, from afar, maybe it does look like one incredibly high typical mountain kind of object, but upon close inspection of its behaviour, it could certainly be revealed that it behaves differently.
Another option is that the "mountain" grows by crystalisation, maybe aided by the natural day-night cycle and special atmospheric phenomena. Think of it a little bit like constructive 3d printing. The mountain grows by deposition, day by day, in a crystallised profile that allows it to grow so tall.
Finally, you can push it even higher by relaxing the specification that the mountain is homogeneous. Maybe it's a combination of the above mechanisms. Maybe the first 15km are rock and another 20km is a "composite tree society" and for another 10km is crystallisation.
Bringing something like this into the story makes it also more flexible, it gives you freedom to talk about other things as well. Maybe the "mountain" "wobbles" continuously because of earthquakes or the flow of wind around it, maybe it allows very fast communications by tapping to its structure.
Hope this helps.