My answer is going to go outside your box a bit so you can see a wider picture. But, before I begin, let me point out...
You're thinking that the past is independent of the present. That you, today, have the ability to make a decision independent of what has actually happened in the past. Well.... maybe you are... maybe you're not.... You need to decide which it is.
Theory #1: There is only one timeline.
The past has already happened. It has already accomodated whatever nonsense you try to do. The reason you can't kill your grandfather is that your grandfather has already survived whatever you try to do to kill him. From this perspective, you exist due to the already resolved causal consequeneces of your future actions. Do you remember that family story about Grandpa miraculously avoiding the only boulder that ever rolled off the mountain and into the road in recorded history? Yup. Cheers.
Theory #2: The timeline is constantly recreating itself.
Let's assume that every infinitely thin moment of time is independent of all other time such that whatever you do to kill your grandfather never affects you because your past is no longer malleable. When that boulder lands on your grandfather (ouch), it happened to someone who wasn't your grandfather because that instant in time is independent of your instant in time, forcing time from the perspective of your late grandfather to recreate itself into a parallel timeline that has nothing at all to do with you. Of course, this means that when you travel back yourself there is no longer a future for you to travel to because the infinitely thin moment you entered instantly recreates the timeline around you. This is known as a one-way trip.
Theory #3: The Conservation of Causality
Or, we can assume that time must instantly snap itself into the shape forced upon it by a changing past. This is the "kill grandpa, pop out of existence" theory. The cool thing about this one is the plausibility of the butterfly effect because what seems like a few meaningless lives (grandpa, dad, me) are actually lives interwoven into thousands of other lives that, themselves, are interwoven into thousands of other lives and the entire chain will be modified because the three of you aren't where you were to begin with. The parabolic hurricane from the butterfly's wings.
Theory #4: Time is an illusion and time travel will never be possible.
Finally, there is no timeline. There is only the infinitely thin moment of time RIGHT NOW! There is no future, only a predicted future. There is no past, only memories and records of things people did. There's nothing to go back to, and therefore there's no way to send a boulder back to off poor old grandad. Time is nothing more than a mathematical concept that gives structure to the chaos surrounding our moment-by-moment lives.
There are undoubtadly other ways of looking at the effects of time travel, but this will do for now.
In your case, you're kinda looking at Theory #1 and wondering, "why can't theory #1 be like theories #2 or #3?" Well.... because it's theory #1.
By the way, all time-travel theories are unproven and will remain so until someone discovers how to affect a moment in time that isn't right now. (Don, why is my glass of water boiling? I don't know Linda, but hey! Why don't we try to affect the past by causing a small space to heat up?) Without that ability, there's no way to actually prove anything about time travel — even mathematically. We can play games with the concept, but to quote the movie Volcano, "Certainty is a big word."
Which was a fun but somewhat lengthy way of saying, "there isn't a way to explain how an event 100% assured to happen didn't happen without breaking any laws."
Of course, an astute philosopher might add, "what laws?"