My search for numbers to support any conclusion to this question that included wind factors led me down a rabbit hole of interesting science. I'll try to keep the following answer as clear and concise as I can.
I started with a basic question: What are the limits of a wall? After some finagling of my Google search terms, I found what must be the most authoritative source of engineering formulae I've ever had the (mis)fortune to try to understand. This report on the Strength of Masonry Walls Under Compressive and Transverse Loads was both an eye-opener and informative, but incredibly dense to the point I spent over an hour trying to understand the equations and what they were telling me. (I'd relate them here, but there's a simplification later, so you can peruse if you want.)
After seeing the term "cavity wall" in that report, I decided to do some digging on what kinds of walls were out there and what their limits were. That led me to a Study on Stress Performance and Free Brickwork Height Limit of Traditional Chinese Cavity Wall. This report indicated that traditional Chinese cavity walls could survive a 6.0-magnitude earthquake if they weren't more than 12.79 meters tall and they could survive a 20-meter-per-second wind if they weren't more than 7.5 meters tall. (Note: handy tool for calculating wind pressure.)
But what about other kinds of walls, like a solid wall? Back to the drawing board. Looking for the limits of a structure, in general, led me to this question on our sister site, Physics SE: How high can be tower or building? (sic) The OP's research led them to a simple equation:
$$
h = \dfrac{\sigma}{\rho g}
$$
The OP did some additional research after asking the question, which produced another equation that for shapes other than a cylinder or cone, $\sigma$ is constrained by
$$
\sigma \geq \dfrac{\rho g V}{S}
$$
where $\rho$ is the density of the structure, $g$ is acceleration due to gravity, $V$ is the volume of the structure, and $S$ is the surface area.
But wait, there's more! From comments on that question, I made my way over to this answer to a question about ice walls. There, the answerer indicated that
[t]he most heavily solicited cross section will be the one at the very bottom, which will be supporting a compressive pressure of $\rho h g$, where $\rho$ is the density of the ice, $h$ the height of the wall, and $g$ the acceleration of gravity.
Comparing that resulting value to the compressive strength of the material in question will indicate at which point the wall will fail. However, s/he also noted:
As an aside note, if you are willing to sacrifice perfectly vertical walls, having a wall with width growing as $A e^{by}$, where $y$ is vertical distance from the top of the wall, will have every cross section of it standing the exact same compressive pressure.
This would allow you to make the wall as high as you wanted.