It would not work.
Using lighter weight gases to provide lift is based on buoyancy. The lighter gas is less dense than the air around it, so the air around it pushes it up just like the denser water pushes a boat up to the surface. The theoretical limit of this process is to use the lightest fill possible: vacuum. At this point, you get an amount of lift equal to the mass of air displaced.
Air is roughly a kilogram per cubic meter (1.225 kg/m3, actually, but we can round to make the visualization easier). This means that a typical 10kg cinderblock would require 10 cubic meters of displaced air. The numbers are actually worse than that, because you also have to account for the material containing the light gas or vacuum, but in the case of building materials, the numbers are clear enough even without this correction. Here's a sense of scale for that:
Now lets consider a typical skyscraper construction, using steel. Steel girders can vary in mass a lot, but the kind they use in skyscrapers is roughly 50kg per linear foot. Thus a 100ft tall building (roughly 10 stories) would need a girder 100 feet long, or 5000kg. That's 5000 cubic meters of gas to lift. That's reasonably close to the fill of a Goodyear blimp, and that's just to lift one girder:
The largest airship to ever fly was the Hindenburg, with 200,000 cubic meters. That's enough to lift 200,000kg of mass into the air. Hindenburg obviously lifted less because its fill was hydrogen, not pure vacuum, and it had to account for the mass of its skin. Actual cargo/crew mass was about 10,000kg, 1/20th the theoretical values, so it could have lifted two of those 10 story girders.
So what if we were serious? What if we threw all limits to the wind, and just decided to lift one of the World Trade Centers (before they collapsed). Records show each tower massed about 450,000,000kg, so 450,000,000 m3 of gas. That'd be 0.45 cubic kilometers of gas. That's not a large area for nature, but that's a tremendous size for a human construction (the Hoover dam comes in at about 1/200th that volume of concrete). That's just to lift one building!
And all of that is ignoring the structural challenges that come with such lift sources. It's ignoring all of the mass of the skin and wiring needed to actually support things. If the Hindenburg's ratio of a theoretical 200,000kg to actual 10,000kg is scaled up, that gas envelope would have to be 9 cubic kilometers in volume!
How big are these numbers? Well, even using the theoretical rate, you'll be forced to use either vacuum (which requires very heavy skins) or hydrogen. Why not helium? Modern estimates are that there's only about .25 cubic kilometers of helium left in North America (deep in natural gas deposits). You would literally use up a good chunk of the world's remaining helium to do the job!
And thus, we stay on the ground. If you really want to leverage lifting gasses in this way, I'd recommend a shift in point of view. Rather than starting with a city and trying to lift it, start with a dirigible and try to figure out how to adapt the concept of a city to work within it. It's much easier to change how people think about cities than it is to lift a city in the air.