Here's a stock photo of Edinburgh. As you can see, there are a lot of fireplaces.
(From www.dreamstime.com's royalty-free section). I used this photo as it is royalty free, but most tenements are in straight rows, not as messy as this
A large part of the housing stock in Edinburgh is what we call tenements. A fairly typical tenement consists of a stair, with 3 flats off each stair, and 4 floors, so about 12 flats total. Each flat has 2 to 4 rooms of 8-15m2, and each of these rooms has a fireplace. There are also smaller box rooms or cupboards which do not have fireplaces, many have been converted to bathrooms. Every flat has at least one 1m2 cupboard which would have been used as a coal store. They vary from quite grand in some areas of town, to quite small and cramped in others.
So this "typical" tenement has 36 fireplaces, slightly more than one per person living there in modern times, though in the time frame you mention, there would likely have been more people in each one. They aren't exactly what you're looking for, but they sound pretty close. They are also very characteristic of Scottish cities, and I have not seen anything quite like them in the US where I assume your story is based. The Brownstones in Boston look similar, but lack the forest of chimneys and also presumably the fireplaces.
These tenements were mostly built between about 1700 and 1850. By 1920-1940, they were often a bit dilapidated, and potentially the cheaper end of the market (many are now very expensive). It would be quite believable to find tenements still using coal in that period, with gas fired heating taking off after the war.