One possibility is for the pushmipullyu to have developped a single alimentary orifice.
Essentially, the second or "posterior" head takes care of eating, the fodder passing down into a complex polychambered stomach, where after much exciting chemistry takes place, the posterior head simply regurgitates and coughs up the compacted stool.
You might be thinking, "oh, yuck! How appropriate is thát for a children's book!?" But in reality, the pushmipullyu is quite polite about such things, and really hardly notices any yuck factor at all.
For you see, the intelligent mind of the pushmipullyu is contained in the forebrain up in the primary head. It is here where breathing and speech happen, and is thus the "social focus" of the pushmipullyu. The hindbrain is quite small and relatively undevelopped. Its focus is entirely on socially acceptable eating, drinking and crapping.
In normal pushmipullyu society, it is very common for a herd segment to gather in a circle, everyone's primary heads can be seen and folks can easily talk and be herd. The secondary heads being outside the circle, can be left to do their essential functions in peace and away from the social functions happening up front.
As for organ arrangement, the pushmipullyu isn't so radically different from other equids. Its fore thorax contains two lungs and the heart, bronchus, larynx, trachea and lesser esophagus. Its hind thorax contains the greater esophagus, female genitalia, uterus and bladder. The central abdomen contains all the organs of digestion.
A note on the lesser esophagus: this structure is thought by natural philosophers to be a vestigial organ, or "fossil appendage" which demonstrates the pushmipullyu's relation to other animals. It does continue to perform a social function, even though it is never used for eating. There are certain times when pushmipullyu herd segments will gather for certain rites, one of which is First Draught. At times during their long migrations, herds will inevitably stop to rest near water sources. The First Draught is quite the social event, as scores of herd segments all approach the water with primary heads bowed reverently, and take a long refreshing drink of water through their usually talkative mouths!
Anatomically, the sphincter connecting the lesser esophagus with the first stomach is very powerful. The reason for this is it must remain closed while the pushmipullyu is expelling its fully digested pellets. It would be terribly rude and would most certainly not do to accidentally expel pellets all over one's interlocutors! As it happens, there are illnesses the pushmipullyu may fall prey to which cause a laxening of the muscles in the lesser esophagus. For that reason, such afflicted individuals are compelled to remain at a distance from their accustomed herd segment.