I've read it somewhere saying that even "do nothing just watch" would change the past. It doesn't obey the physical law because the person who travels backwards in time absorbs light and thus is interacting with the past universe and therefore changing it. 

I don't quite understand this. Could someone explain it? Thanks!

(There is only one single timeline of course. I'm not going into a parallel universe)