No
Obligate bipedal ungulates are extremely unlikely to occur (evolve) naturally for a very clear reason.
The Problem Of Speed
Goats, deer and other such medium sized ungulates can stand (and also walk for short distances) on two legs. They usually do this when trying to reach higher foliage of trees. Several species of mountain goat also tend to get bipedal during male head-butting competitions.
However, there is no exception to the rule that bipedal ungulates walk very slowly on two legs as compared to their speed on four legs. You would never see a deer running for its life on two legs.
But what about the ungulates which don't need to run in front of predators? Buffaloes and bison; these large ungulates use their massive heads for their defense. When standing on two legs, they would be unable to use their heads for defense and they would also be exposing their vulnerable underbellies to the predators. No, that is not going to work.
Some might argue that once ungulates turn obligate bipeds, their physiology can evolve for higher speeds. That is not a valid argument. In order to evolve for higher speeds, these ungulates must survive long enough (a few hundred thousand years at least) to first become compulsory bipeds. Currently the life expectancy of a compulsory bipedal deer is less than a day. It does not seem likely that tigers, cheetahs and leopards would cherish the sight of a clumsy, bipedal deer enough to not invite it on their dinner tables.
The morphological changes from quadruped to biped stance are immense and include (but not limited to), changes in the backbone, in the skull features (specially where the brain connects to the spinal cord), in the leg joints, in the heel joints, in the shoulder joints and also in teeth and jaw structure.
Why Humans Survived The Change And Ungulates Wouldn't?
We humans made the successful transition from quadrupedal to bipedal gait but ungulates cannot. The simple reason is that we humans never relied on speed to evade (or defeat) predators. Since the time of Sahelanthropus (7 million years) to modern humans (0.2 million years ago), we have always relied on our superior brains, dexterity of our hands and supreme communication skills for survival.
On the contrary, ungulates have neither of these. Their mental skills are by no means any superior to their predators, they cannot even pick up a twig from the ground with their hands and they cannot even count from 1 to 5. During our early evolution, when a sabertooth attacked our great great forefathers, it only took them one vertical leap and some quick hand motions to reach the shelter of the branches. No such option is (or was) available to the ungulates. They would not able to make it.
Remember the terror birds of South America? They were bipedal and they were much better runners (like most flightless birds) than humans (we have evolved completely for bipedal gait and our running speeds are higher than any and all other bipedal mammals). Yet those awesome carnivorous birds were exterminated by mammalian carnivores of North America. How much prayer can a bipedal deer have against a ravenous cheetah?
The Incentive Of Height And Why It Would Not Work
Another (purely theoretical) argument for evolution of compulsory bipedal ungulates can be that of height and reach. Clearly, a bipedal deer would have greater reach than its quadrupedal cousin. In times of famine this could turn into the decisive variable between starvation and survival. If the famine lasted for a long time, a species of obligate bipedal ungulate could evolve.
This assumption is based on faulty logic. First, our planet has been through some seriously severe famines during its history and none of them has resulted in bipedal ungulates. Second, most ungulates are grass-eaters, not foliage-eaters and their diets primarily consist of grass species. Third, even if somehow some species of bipedal ungulates do evolve, they would be quickly devoured to extinction (due to slow speeds) by the carnivores once the famine ends and carnivorous populations re-establish.
Instead of bipedalism, mammalian evolution for longer reach has always resulted in longer necks. The giraffe and camel are the best examples for this.
Conclusion
No, bipedalism is extremely unlikely (we never say never in science) to evolve or have evolved during the history of mammals. The only time it could have evolved and remained was during the origin of mammals, some 221 million years ago. However, once the general mammalian body shape was established firmly by late Jurassic, the transition from quadrupedal to bipedal gait was too risky and dangerous (not to forget without any real rewards or incentives) for any mammalian group.
Humans were able to successfully transition from arboreal to obligate bipedal lifestyle because we never relied on speed for survival. We had higher intellect, dexterous hands and superb communication skills to help us win against predators. Current (or prehistoric) ungulates never had these abilities to bail them out against predators, so they cannot survive the transition.