Short answer is yes, they would. Longer version as follows. I've filled out more detail to accommodate the hard science tag.
A rocket engine basically works through gas expanding and being thrown out the back of the engine at high speed. What that means in practice is that as the gas expands as a result of the fuel being burned, it pushes out in all directions pretty much equally. What the bell housing on the back of the engine does is give the gas one direction to escape in, pretty much pushing the engine forward.
Now, as it turns out, this is nowhere as simple as it sounds, and there's an issue called Optimal expansion that defines the efficiency of the rocket in terms of how the rocket expels its gas, thanks to the nature of its cooling down immediately after the reaction.
The nature of that effect means that the most efficient design of nozzle is going to be the Bell Nozzle, which is a parabolic curve that caters for the expanding gas, the fact that some of the gas doesn't escape in exactly the right direction, and correctly 'shapes' the exhaust. These are the hardest nozzles to fabricate (and the most costly), but are used on anything where weight is a factor, meaning large rocket engines like the Saturn V, Space Shuttle, etc.
What the bell housing does is shape the exhaust as it escapes, guiding all the angled releases so that they go backwards in pretty much a uniform direction, even accounting for change in the exhaust temperature and pressure. As such it keeps the engine thrust directed cleanly and increases the efficiency of the engine by minimising the amount of gas that pushes in an oblique direction to the desired thrust angle.
A square housing won't do that.
The simplest answer as to why is that if the gas is expanding and being released in a common direction, the gas will expand radially, forming (in essence) a circular impulse wave. In effect, that means that the pressures on the inside of the square housing are now uneven, causing pressure to build up, especially in the structurally weak corners of the housing, greatly increasing the possibility of failure just like in aircraft windows.
Because all gas escapes from the centre of the bell housing, a round housing just automatically shapes the errant vectors in all cases. A square one will only do so in some instances, namely in the middle of the sides and in the corners. In all other cases, the side of the housing is tangental to the oblique gas vectors, creating inefficiency and (potentially) misaligned spurts of thrust from the engine output, making it less accurate as well.
The reason this is a problem is because of the chamber pressure. On the Saturn V, the F1 engines had an operating chamber pressure of 70 Bar, or 70x the pressure of the atmosphere at sea level. That may not sound like much, but then you have a massive amount of gas wanting to escape, and that goes out the nozzle, and into the bell housing, putting pressure on the internal walls of the nozzle. As has been pointed out by Mark, when you have tangental vectors being generated is that the the gas can be directed into those corners in slightly higher concentrations, and corners are a natural stress point as you highlighted above. This means that your thruster is less likely to survive the stresses of engine burn than a round bell housing is.
As a final point, as has been documented in comments there is a 'kinda' square engine possible in the form of Aerospike Engines which may well fit your bill. These are often described as 'inside-out' engines because of their design, but they may well be a solution to your need for square looking nozzles. Certainly, from a hard science perspective, you could simply declare your ships use this type of engine and move on; they are a real thing (although not in active use as yet) and they have as close to the shape you want as I suspect you'll get.