Magic is a form of sorcery that involves using the unlimited mana of the world around you (Gaia) and bending it to your will. Ritualized magic is essentially passive and formulaic, with the individual "borrowing" the world's mana, feeling it flow through them and guiding it along to perform a spell. It is a slow and patient process, with the magic being derived largely from the ingredients and the rituals of preparation. It is very similar to following a recipe and may be affected by the quality of the ingredients and the precision of the ritual. Short rituals are relatively easy to perform, simply requiring a ritual circle, few components, and easy to memorize. longer rituals are more complex, requiring hours of chanting or movement, and more ingredients. These steps are meant to control the flow of magic into the real world, and coax the energy into a specific form, similiar to cooking.
Wild magic is much more aggressive than its ritualized form. It involves dominating the world's unlimited mana and subjugating it to suit your needs in order to power a spell. It does not require ingredients or special preparation, only simple desire. Because of this, it is far quicker and more powerful. This is a taboo form of magic because it is seen as taking what isn't yours from the world for a selfish purpose. Nevertheless, mortals are tempted to use it because it is much easier to use and provides a quick avenue to power.
I want wild magic to have some consequence to the user and the surrounding area that builds up over time the more it is used. Why would this form of magic negatively impact the people who use it as well as the surrounding environment?