The Roman Empire was very successful, but perhaps its failings started right at the beginning
Considering most civilisations throughout history, the Roman Empire is often considered among the long and powerful. However one should remember prior to being an Empire it was a Republic.
Many books and theories have been written on this subject as it is of supreme interest to all who want to know how civilisations collapse. In essence though, the Roman Empire 'decline' was not an instantaneous event, but rather a long period of many centuries of all sorts of different reasons. Considering that the 'decline' lasting centuries is in fact longer than some civilisations whole existence, and spanning many hundreds of generations, it is remarkable: it was so gradual at the time you would hardly notice it.
The dissociative aspect of these reasons, too, lead many to the conclusion that a civilisations 'decline' is inevitable: Soon all civilisations would encounter one too many challenges and collapse.
However consider this: the Roman Empire was 'established' in 27BC, considered by many to be the acquiescence of the Republic to appoint Octavian as Emperor. The greatest extent of the Empire territorially is 117AD, barely 145 years later, being only slightly larger than the previous Republic.
On your definition, the 'decline' using territory as a measure, therefore was gradual after that, for another 359 years till 476AD (By the way, you regard Roman Empire as the Western Empire, whereas the Eastern Byzantine Empire lasted till 1453AD). This means the 'decline' according to your definition began only shortly after its 'rise'.
In contrast, the Republic preceding the Empire, although tumultuous, lasted 362 years and for most of this period was expanding.
Looking at this alone one has to wonder not what would have happened if the Emperors continued to rule, but what the world would have been like had the Republic continued? After all, the Republic was expansive, adaptive, and oddly resilient: It even survived a sacking of Rome and many many invasions.
What the Empire ultimately lacked was the ability to adapt to altering conditions: Economic, Religious, Political and Monarchial pressures which all contributed to the inability for Emperors to govern wisely and comprehensively.
The best Emperors had trouble (prior to being assassinated), the worst Emperors didn't care (prior to being assassinated). It is worth noting over half of emperors were assassinated, the remainder forcibly deposed and then killed (or unknown cause). Constant infighting and splitting of the Empire occurred between rival Emperors. Perhaps all that was needed for the 'Empire' to continue was for an Emperor to discontinue being an Empire, and convert back to a potentially more demonstrably adaptive Republic instead.
Note that many view the 'decline' as not much more than a political change on a map: The Roman civilisation was so successful its structural, social, infrastructural, economic and philosophical foundations were still in use till up to the present day.