While it might be tempting to design birdhouse-like structures, consider that civilized creatures (i.e. not struggling for survival against elements and predators) will probably be lazy. And flying is a lot of work. And if they're anything like humans, there will be hoarders who want to move lots of heavy stuff into their homes. Also consider elderly and disabled, which isn't a thing in the wild because they die, but would be a real concern for a civilized species.
I would think that maybe the architecture would have birdhouse-like impressions to it, but that there would probably be ramps or pulley systems for goods, and likely walkways between levels. Also, keep in mind that building vertically will always be more expensive than building flat, because vertical structures require more design and stronger materials. Although, for these residents a story is probably only 6-7ft and it doesn't need to hold nearly as much weight. unless they like pottery.
To imagine a city of these people, I'd guess that the wealthier residents live in artfully designed multi-story houses with stone/earthen bases and wooden pole construction for 3-5 stories. Less wealthy residents probably have 2-3 story dwellings, and maybe the poorest residents are stuck making nests in the hills around the city, with the poor elderly and/or disabled ending up at the bottom of those hills, and young "street rats" ending up at the top of the hills.
The homes would probably have very small windows, or barred windows, since any large opening could essentially become a door for a burglar. Walled garden type things wouldn't probably exist since the wall would serve no purpose. All angles of the home would be visible, so if they had stuff they didn't want seen it would need to be inside or underground. All homes will likely have large sturdy roofs, to protect against falling objects (imagine the common occurrence of accidentally dropping goods or equipment from several hundred feet up, or those weekend parties when everyone is flying home drunk and can't be bothered to find public toilets) Lightweight creatures probably don't have endurance for too much digging, so basements are probably rare.
For moving the quantities of food needed for a city, boats are probably still the most efficient method, so a sizable city would probably still be along the water. That said, a creature that can move so much faster than a human might be able to fly out to nearby farms for meals, so they wouldn't be as reliant on supply chains like humans are. But, don't underestimate the space needed to grow enough food for the population of a city.