The question is difficult (impossible) to answer both because no scientific data exists, and because of how it is worded.
There exists some data which could be extrapolated, but it has a small sample size, was collected from a set of carefully chosen individuals, and the effect is intermixed with other effects (e.g. radiation, stress, air composition, nutrition). Thus, it is not possible to generalize to "nearly all humans", nor to blame any observed effect on gravity without the possibility of other factors also having an influence (unlikely as it may seem, but truth is we simply don't know, we can only assume what's likely).
Reduced gravity most probably predominantly leads to bone demineralization and muscle atrophy (and possibly other things). That's because this is what people who are (among some other effects) exposed to reduced or zero gravity demonstrably develop. There may of course be other reasons for that effect, but gravity is the likely factor.
Exercise can at least partially counter these effects, but exercise is not a premise of the question.
If, what's very likely, gravity is indeed the deciding factor (and not e.g. NASA's bad nutriention scheme) simply because less "pull" means less stimulus, then the effects will likely occur at every significant reduction. Whether or not it counts as "not tolerable" is up to your decision, so it's fundamentally impossible to provide a number. Is it tolerable to have 5% less strenght and bone density? What about 10%? Who can tell. For a boxing champion or a rugby player it sure wouldn't be. For a computer geek? Why not.
Seeing how "return to normal environment" is not a premise of the question (only "compared to" is), though, one might as well say "zero gravity is perfectly sustainable" since you do not need (shouldn't need?) strong muscles and hard bones in zero-G. Basically, it would mean: Yes, there will definitively be serious changes, but they aren't a problem.
The same goes on the other end. Something like 4-6G will without doubt cause noticeable discomfort, and rather soon evolve into problems. However, 1G or 1.5G is not really much of a difference, nor is 2G. Only just, everything is a tidbit (tidbit, eh?!) heavier. You'll grow more muscle and denser bone, tripping and falling will hurt considerably more, and you'll need more energy. It'll be generally harder, your heart and your circulatory system needs to support higher pressure, but all in all, there's not really that much difference. Sure, more load on your spine, your knees, and on your inguinal canal. All in all, that's not precisely enhancing durability or longeviety.
But there is no real reason why you couldn't live in 2G for 5 years, if you are reasonably healthy. Mind you, there's people who have 150-180 kilograms of body weight (or more?!), and some of these carry that around for 30-50 years. Do they eventually have problems? Well sure, but not after 5 years.
However, "nearly all" includes infants and people who are 95 years old and suffer from heart insufficiency and COPD, so... it may be troublesome to make a claim!
A baby in zero-G will be "fun" when gooey, stinky stuff comes shooting out of it at both ends. Which tends to happen regularly. A "typical" 95 year old will not be very happy in 2G, at least not for long. Someone with vein insufficiency will not be very happy either -- them thick legs getting twice as thick now. Allergic and going to zero-G? Well guess what, sneezing all day long can be a lot of fun!
Speaking of infants, it is not certain in any way inhowfar growth in children (or animals in general) is influenced by low gravity.
Generally, data about animals in space is relatively sparse, and in my opinion the conclusions that are made are dangerously naive. For example it is not appropriate to extrapolate from mouse experiments based on "they have short life spans, so this is a long-time mission for them" as has been done on the ISS. While it is true that their lives are shorter, it does not mean that they automatically experience long-term effects faster because of that. Sometimes, this "logic" holds, but sometimes it doesn't.
Medical research is full with examples of this fallacy. Such as the famous longeviety experiments where a few years ago someone got "immortality, soon!" for humans because he managed to extend the lifespan of some bug from one day to two weeks or so. Which is great, but it's also entirely meaningless.
Acceleration on Earth is constant within what's reasonable to claim. There's 5 Gal difference between the absolute lowest and highest, that's ~0.5%. The difference between what "nearly all" people will ever experience during their lives from mountains or denser ground is in the two-digit milli-Gal range. That is "constant" for all practical purposes.
So, all we really know for sure is that most humans stay mostly functional most of the time at around 980 Gal (i.e. pretty much 1G +/- 0).
And even under these conditions, 5 years are demonstrably not sustainable for some individuals.
Tasks essential for living in a society should not be interrupted by the different gravity. Well, what does that mean!
What tasks are essential for living in a society? As we are presently being forced to experience, going out and meeting people is not essential. Nor is working. Although both would in principle be perfectly possible in either a zero-G or a 2G environment.