For a small impactor, it matters
The angle basically affects the way that the kinetic energy of an asteroid is dissipated. For a small asteroid at a steep angle (i.e. your yellow line, nearly straight up and down), little of the energy is dissipated in the atmosphere, and most is dissipated on impact. That means you get more ejecta; dust and matter thrown into the air. For an asteroid at a shallow angle (green line, parallel to the surface), much of the energy is dissipated in the atmosphere as heat energy. This can be enough energy to cause significant localized heating. For example, a 1 km radius asteroid hitting at 15 km/s has enough kinetic energy to heat the entire atmosphere by 1 degree Celcius; the local heating could be significant.
I would argue that the 'destructiveness' would be backwards from what you are arguing. If the meteor 'grazes' the atmosphere, it will lose most of its kinetic energy before hitting the ground. There will be localized atmospheric heating, but that will dissipate soon, and the overall heat will be sent back into space as IR radiation within a few days. Not too much change for life on Earth.
However, the 'up and down' impactor, will deposit most of its energy in the ground. The 1km asteroid I proposed will release about about 10 times more energy on impact as Mount Tambora released on its 1815 explosion. This was the explosion that caused 'the year without summer.' So, in this case, you would rather have the heating in the atmosphere, than the ash and dust clouds causing a volcanic/meteoric winter.
For a large impactor, it doesn't matter.
The Chicxulub impactor, which killed the dinosarus, had about 100 times the energy of the 1 km impactor that I proposed in the last section. If it had deposited all its energy in the atmosphere; it would have heated the entire atmosphere to about 110 C and driven much of the atmosphere off into space. But, a meteor that large won't barely be slowed by the atmosphere before it plows into the Earth. In this case, only a tiny fraction of the energy will be lost to the atmosphere, no matter what angle it hits at. It is going to hit the surface and deposit its energy there. Remember, even being 'slowed' by hitting the ocean, the Chicxulub impactor left a crater hundreds of km across on the ocean floor.