In historical illustrations of cities with defensive walls, I have noticed that it is very rare for a non-defensive structure to be built directly adjacent to the inside face of the wall. A building constructed adjacent to the wall would have one of its external faces already built, a potentially huge saving, so I imagine there must be some very good reasons to keep the inside of a defensive wall clear.
It was done. For instance, the French settlement of Louisburg in Nova Scotia had building in some places along the walls, as seen in this photo of a diorama of the fortress:
As for the reason why it wasn't common, open space inside the walls offers an advantage: if the enemy breach the wall, they have to cross an open killing ground where they are exposed from fire from both inside and forces still in the walls. It allows troops on the inside to quickly move around en masse without having to move through narrow streets or buildings.
From a non-military standpoint, it provides open space that can be used for festivals, for pasture, for parade grounds, and even simply for parkland.
Ideally you want a dead zone on both sides (inside and outside) of your walls, both for purposes of spotting enemies in that area and for logistical purposes (quickly getting troops and supplies to the wall, and as a patrol zone).
Structures built up to the wall block that, which is why they were typically discouraged unless maybe very easy to take down (like a market stall would probably be ok in peace time, but removed during a siege, a blacksmith shop wouldn't be).
Of course many cities expanded over time, with new walls being built outside the old ones, and the old walls then no longer needing to have those dead zones so things would be built right next to them, even using the old city wall as part of their structures. Or the city fathers may decide those old dead zones make a nice city park and circle road, which both preserves them for posterity and gives them a new purposes and has the added effect that the wall now can again be put into service as a secondary defensive structure were the city to fall under siege again and the (new) outer wall be breached.
Why you wouldn't want buildings next to walls
If you put houses next to the wall, how are your soldiers supposed to run around the perimetre? You're gonna say "on top of the wall", but on top of the wall can be very risky during a siege. It's certainly more practical and safer to move your troops behind the wall than on top of it.
In peace time, having a street that runs around your city, like the modern périphérique of Paris (which replaced ineffective defensive walls), is kind of nice to have. It makes transporting people and goods a bit easier, although how much of a benefit that is would depend on the size of your walled city.
Buffer zones are nice. If the wall is breached, it might be helpful to have a bit of space to defend inside your walls, compared to fighting in somebody's garden or living room.
Don't be a target
Related to the above. Suppose the military let you, would you build your house next to the fuel depot of an air base? In case of an attack, your house would be blown first. When the wall is breached, you don't want enemy soldiers to come through your living room. It's not a lot more comfortable to have them come through your backyard, but it's a mild improvement.
Space isn't an issue
Usually the limit to how many people can be sustained inside a fortified city during a siege isn't the amount of real estate, it's how much water and food you can store/get. So realistically, you don't need to fill all the space. If you did, that would be woefully unsustainable during a siege.
Why it's fine to have building next to walls
There's always a bigger wall
The Cité of Carcassonne has buildings rather near the walls. That is, near the inner walls. Carcassonne has two lines of walls, with no buildings inbetween. That provides the buffer zone in case the first wall is breached, and if you're anticipating enough sieges that you're wondering where you should build your buildings, start by building a second outer wall first.
Let the Wookiee win
The truth is sieges favour the attacker. Unless you have an army outside coming to help you, breaking a siege is really hard, and then it becomes a game of waiting for the defenders to starve out, helped by a few choice artillery hits.
It was well understood that the longer a siege lasted, the more pillaging, raping and murdering the winning attacker would deal. A prompt surrender usually avoided such a fate, and it was better for the attacker to capture an intact city than a pile of rubble.
Of course, it was also understood that if the attacker had a reputation for being a bunch of savages, you'd never surrender. But when attacked by a reasonable enemy, surrendering quickly isn't a bad choice. And at that point, it doesn't really matter where your buildings are.
Fortified cities aren't built in a day
What might have once been a carefully planned city ebbs and flows through the centuries. They're conquered, retaken, lost, abandoned, restored, each new occupant adding or removing buildings, time bringing in different styles and techniques. The Cité of Carcassonne, to keep the same example, in some form or another is over two thousand years old. It has been a prosperous city, and it has been a ruin of squalor. Today it's a tourist attraction.
Military consideration aren't always the order of the day.
Different people built different castles
There are different philosophies regarding forts, not just through time but also through space. While they have a number of commonalities, German and French medieval castles for instance have their differences. Because of different architectural styles, because of different cultures, and also because warfare evolved through times and at different paces, and because each particular location has its own history.
Each fort has a way of being unique.
In medieval warfare, an important aspect of siege tactics included escalade, or the scaling of city walls. Having structures immediately adjacent to defensive walls would enable the sort of "hiding spots" or "blind spots" that would be prone to attacks or laying-in-wait, or holding positions during an invasion, which would serve little defensive purpose but would pose significant hazards for the defending party.
Also consider that most tall structures simply did not have any need to be on the edges of a walled city. Most would be located in the center area, and any purpose that needed to be fulfilled by a tall structure could simply be merged into a tower in the wall itself. There's really no need to have skyscrapers sitting around in the edges of one's city, after all.
The ancient world did things differently, it seems.
The Old Testament has multiple references of people whose houses were on the city walls. The most famous by far is Rahab, who let two Israelite spies out her window to escape. I don't know when the changeover happened.
Joshua 2:1,15-16 CSB Joshua son of Nun secretly sent two men as spies from the Acacia Grove, saying, "Go and scout the land, especially Jericho." So they left, and they came to the house of a prostitute named Rahab, and stayed there....
 Then she let them down by a rope through the window, since she lived in a house that was built into the wall of the city. "Go to the hill country so that the men pursuing you won't find you," she said to them. "Hide there for three days until they return; afterward, go on your way."
For every stone that needs inspecting and repairs, there needs to a be a space
Three feet in each direction and six feet tall.
Any higher than that and it needs Putlog holes, which do what it says on the tin, so you can put scaffold.
(1) Depth of Working Space
The depth of the working space in the direction of live parts shall not be less than that specified in Table 110.26(A)(1) unless the requirements of 110.26(A)(1)(a), (A)(1)(b), or (A)(1)(c) are met. Distances shall be measured from the exposed live parts or from the enclosure or opening if the live parts are enclosed.
(2) Width of Working Space
The width of the working space in front of the electrical equipment shall be the width of the equipment or 762 mm (30 in.), whichever is greater. In all cases, the work space shall permit at least a 90 degree opening of equipment doors or hinged panels.
(3) Height of Working Space
The work space shall be clear and extend from the grade, floor, or platform to a height of 2.0 m (6 1/2 ft) or the height of the equipment, whichever is greater. Within the height requirements of this section, other equipment or support structures, such as concrete pads, associated with the electrical installation and located above or below the electrical equipment shall be permitted to extend not more than 150 mm (6 in.) beyond the front of the electrical equipment.