Step 1: maximizing planet size
Having the largest potential body gives us the most space to work with.
I'm going to assume a rocky planet because gases generally don't form mountains very well, and massive wind speeds will work against our goal. Wikipedia directed me to this paper, which suggests that 1.75 Earth radii is the upper limit for rocky planets. 5 Earth Masses is the round number floating around this size of planet, which gives us a surface gravity of about 1.6g.
Step 2: building a mountain
I'm going to run with the idea of a shield volcano, since that category includes the largest mountain in the Solar System and the largest base-to-height mountain on Earth. According to wikipedia, these are usually pretty shallow, with a typical height/width ratio of 1/20. Olympus Mons on Mars is steeper with an about 1/11 average slope, but it only has to handle 0.4g instead of out mountain's 1.6. I will be running with 1/25, because I can assume some optimization on our lava composition and don't know how I would calculate the exact ratio
But how wide can we make the mountain? Since the layers form in a liquid state, I think it's reasonable to assume that the shape can be scaled up without breaking. In this case, we are limited by the size of the planet, since after that point we are just increasing the planet radius. In other words, our maximum width is half the planet's circumference, and our maximum height is 1/25 of that, or 1401km.
Step 3: minmaxing
The tallest mountain on Earth by your criterion is neither the tallest base-to-height mountain, nor is it the mountain with the highest altitude. This is because the Earth's rotations cause the shape to be squashed such that the equator is farther out. There doesn't seem to be data on how fast a large rocky planet can spin, and the actual effect is hard to calculate because planets have a non-uniform composition, so I'm going to assume that we manage to get the same flattening as Earth (1:300), and position our globe-spanning volcano on the equator. This isn't a large amount, but it'll add a couple extra meters.
result: 1413 km
Note that this is not a peak by any stretch of the imagination, it's a very shallow bulge that takes up the entire planet.