Put it farther away from the sun (or use a dimmer sun) so it doesn't boil, and fill the atmosphere with lots of carbon dioxide.
CO2 is considerably denser than O2 or N2, and so easier to float a balloon in. But it is also extremely common, unlike SF6 or heavy noble gasses. And, it is actively toxic to humans at concentrations not very much higher than what Earth has, causing hyperventilation, respiratory acidosis, and reduced cognitive function, even when there is also sufficient oxygen to support human life; i.e., it does not merely cause death by suffocation.
There are two planets in our own solar system with primarily-CO2 atmospheres (Venus and Mars), one of which is extremely thick (Venus), so forming such an atmosphere naturally is clearly easy to do--unlike naturally accumulating huge quantities of SF6 or xenon.
More exotic options:
Hydrogen Sulfide (H2S): It's just slightly heavier than air, and it would react with oxygen fairly easily so you would need a mechanism to continuously replenish it if the atmosphere is meant to otherwise support oxygen-breathing life, but it is acutely toxic at even very low concentrations.
Sulfur Dioxide (SO2): Much heavier than CO2, and it's an acute respiratory irritant--it will specifically damage the lungs, and eyes, and at higher concentrations irritates the skin. Eventually, it causes bronchial inflammation, and pulmonary edema, and laryngeal spasms. It can react with oxygen to form SO3, which is a solid at human-survivable temperatures, but not as rapidly as H2S is destroyed, and it's easy to imagine biological processes which would destroy SO3 to regenerate SO2. It also reacts with water to form sulfurous acid (a major component of acid rain). The known existence of worlds like Io, which high concentrations of sulfur on the surface, make this plausible.
You could also go with a combination of CO2 and SO2, providing multiple mechanisms to aggravate and poison humans without excluding native oxygen-breathing life.
Nitrous Oxide (N2O) and Nitrogen Dioxide (NO2): in the absence of denitrifying bacteria, all the oxygen in the atmosphere would eventually react with nitrogen to form various nitrogen-oxygen compounds. Nitrous oxide is a sedative, and nitrogen dioxide will react with water to form acids just like sulfur dioxide does, and produce similar respiratory irritation. Hal Clement's The Nitrogen Fix explores the aftermath of a catastrophe in which biological nitrogen fixation outstrips the rate of denitrification, turning Earth into this sort of world. Native organisms might not need to breathe at all, relying exclusively on dissolved nitrates as a metabolic oxidizer, but there could still be free oxygen in the atmosphere as well if photoautotrophs (plants) keep producing it.