"Very close" is a relative term in astronomy. The closest planet to our sun is Mercury, which is 57,910,000km from the sun. I can say with great confidence that flying at a cruising altitude of 10km, bringing you to 57,909,990km is not going to make all that much difference.
What makes it warmer to get closer to the star anyway? The temperature of the radiation reaching you is exactly the same. The only difference is that the star is making up a larger portion of what you can see, so it can heat you faster. You would be looking for the star to be so ungodly close that moving up towards it actually noticeably increases its size in the sky.
To get the effect you describe, I think you would want to have a brown dwarf instead of a star (brown dwarfs are nicknamed "failed stars" because they lack the mass to fuse hydrogen, but can fuse other elements). Brown dwarfs can range from anywhere from 3000 degrees down to the temperature of hot coffee. Then you want it close enough to touch. If you can have a transition from purely radiative heating to conductive heating as you go up. However, you're going to run into issues with drag. Such a planet is on a death-spiral into the sun.
Perhaps the easiest way is to have a substantial cloud layer. If your planet is permanently covered in highly reflective clouds which are transparent to low temperature radiation (like that from the ground), the ground would be quite cool. However, ascend above the clouds, and you suffer the full gaze of the star.