This sounds chaotic
What you're referring to is chaos theory, popularly summarized by the essay title Predictability: Does the flap of a butterfly’s wings in Brazil set off a tornado in Texas?, subsequently known as the Butterfly Effect
A chaotic system is defined mathematically in a complicated manner, but the basic idea of it is that a tiny change in the initial conditions of a system has a vastly different change in the trajectory of the system. It's also important that the potential trajectories all cross each other at some point (otherwise, predictable actions like exponential growth would count as chaotic).
A pendulum on the end of a pendulum is a chaotic system. Moving the pendulum ever so slightly either direction results in a vastly different path that the end of the second pendulum will follow.
Atmospheric conditions are considered chaotic, and this is the reason that meteorologists generally give weather forecasts one week or less in advance. Given that, it's likely true that, as you say, any change in atmospheric conditions could result in a different offspring, or no offspring.
How far away do I have to go to avoid threatening my own future?
If you change something in the past, you likely can't, on the basis of distance alone. You might be able to make the change far enough away that your change will not affect your conception (maybe), but this does not put you "in the clear," so to speak. The chaotic system that is reality will have changed and its future trajectory might or might not include your time traveling antics. Maybe you cause a rainstorm in a different country that causes the automotive death of the brilliant scientist who would otherwise have designed the time machine.
Maybe your atmospheric changes affect economics and someone murders you out of desperation before you can ever time travel (though your son might become Batman, so that would be a plus).
Your only certain chance of protecting the future of your past self in order to avoid creating a time paradox would be to find a completely isolated system which has remained isolated since the time period you intend to travel to, and travel to the inside of it. By "completely isolated" I mean airtight and not resting on the ground (it would need to be in free fall, or in outer space). Since it has been isolated the entire time, it doesn't matter what changes you make to it (unless you use some tricky momentum tricks with a well-timed travel to the past or present to cause it to become unisolated), because should it ever become unisolated, that will be in the future of the moment you left in the present.
But all this depends on a vital assumption: that you are changing the past. So...
Are you actually changing the past via time travel?
I'm gonna paraphrase myself here:
The Novikov self-consistency principle states that any time travel is mathematically required to be self-consistent. That is to say, the traditional time paradox isn't possible.
Consider drawing your situation to its ultimate conclusion. You travel back in time, disturbing the air, which prevents your birth. Traditionally, this is a time paradox. If we were never born, we could have never time traveled and prevented our own birth. There are a number of suggested "resolutions" to the time paradox, almost none of them good for us.
Novikov's solution is different. Novikov suggests that none of this could have happened in the first place. For some reason, regardless of whether it the reason is known for any particular instance, it is simply not possible to create a time paradox, much in the same way it is not possible to create any other kind of paradox.
If I work out a mathematical proof that shows that $1=2$, then there is something wrong with my proof, because we already know $1 \ne 2$. What is wrong exactly? Well I dunno, it depends on the "proof." Being unable to determine the reason does not mean that the paradox is allowed to exist.
In our case, the only way you could travel to the past into the same room with your parents when you were conceived would be if you had traveled to the past into the same room with your parents when you were conceived.
There is a pretty detailed example given at the above link involving a billiard ball traveling into a wormhole with the precise trajectory that will cause it to strike itself and knock itself off course, preventing it from entering the wormhole. Novikov refuted this example by redoing the mathematics himself and finding a number of self-consistent solutions.
Think of it from the perspective of a non-traveling observer. Your parents are working on making you, when suddenly out of nowhere, the adult you steps out of a wormhole. Your parents apparently don't react by stopping (they must be into this sort of thing) but you've shaken things up enough to cause a different sperm to join to an egg than it would have. But this is ok, because as our observer finds over the next decades, the resulting child is you, the person who stepped out of the wormhole that day. In this solution--rather than how we normally think of time travel as messing up the opportunity for time travel, therefore being inconsistent--your time traveling has instead complemented the opportunity for time travel, enforcing its own self-consistency.