2-sex reproduction has evolved multiple times, strongly suggesting that there is a good reason for doing so. As for exactly what that reason is, there are a number of theories. Most likely, all of them play some part in the reason why this seems to be the trend among life on Earth.
The advantage of hermaphroditism is obvious - more potential mates for everyone. So I'm going to mainly focus on the advantages of a 2-sex system.
Self-fertilization is impossible
Self-fertilization reduces genetic diversity, which makes a species more vulnerable to predators and parasites. While many hermaphroditic species today have worked out alternative methods to avoid self-fertilization, like the complex reproductive organs of snails or alternating generations in plants, these mechanisms are complicated. The earliest aquatic life generally reproduced by simply releasing their gametes into the water, and splitting the sexes to prevent self-fertilization was one very common strategy.
Many fungi actually have a method of combining the advantages of the hermaphroditic system with the impossibility of self-fertilization by having tens to hundereds of different "sexes", (or mating types) that are functionally identical except that they can only breed with a type not their own. This increases the chances that any fungus they encounter will be compatible while ensuring they are not compatible with themselves.
Male seekers, female choosers
On the most basic and primitive level, the difference between males and females is that females expend a large amount of energy to gestate a child - whether constructing an egg or in a uterus - while males spend less. This simple distinction creates a number of trends with the benefit of maximizing the ability of good genes to spread.
The most basic of these is that the female does not have to expend much energy finding a mate - the males come to her and often compete for her attention. Instead, the female's role is to choose the most genetically fit male, a role that does not require a lot of energy expenditure. This energy can then go towards gestation and producing strong offspring. This also allows a highly fit male to produce a lot more offspring, while unfit males reproduce little if at all. This streamlines the evolutionary process.
Further specialization of roles
Once a basic male-female split exists, it is possible for males and females to specialize in other, more interesting ways. Among many animals where the male brings a "nupital gift", the female gets easy extra resources from her suitors. Among spiders, the female gets an easy extra meal of her suitors. Among mammals, the male often serves as a protector of the group and will have special adaptations for fighting, like horns or large size. Male birds often use their bright colors to lure predators away from the nest while the drab-colored female remains hidden and protects the eggs.
One interesting trend is that the more monogamous a species is, the smaller the difference between males and females becomes, and the more "typical" roles tend to blur - probably because if the male is spending less energy seeking new mates, he can spend more energy raising the young. Many birds share nest-watching and food-seeking duties, and emperor penguins reverse the typical roles by having the male watch the egg while the female finds food. While humans have some of the typical mammalian "male protector" traits like males tending to be larger and stronger, this difference is far less pronounced than it is in other apes.