The first thing to go is always your memory...
Cold storage
Cool, dry conditions that preserve a computer at least well enough to prevent corrosion and physical damage are a must. In damp or otherwise corrosive conditions, a computer may not last a year; delicate wiring can very quickly decay, especially if there is no moving air to limit humidity inside the case. Temperature can also play a key role; too wide of a range or too rapid of change can stress physical components. If the computer is in a moderately stable environment, above 50 and below 110 degrees Fahrenheit, it has the best chance of lasting a long time. Sunlight - or even low ambient light - can destroy almost every part of a computer, so it should be stored somewhere out of the light. If a computer isn't stored somewhere cool, dry, dark, and free of vibrations, expect it to last a few years; the worse the conditions, the lower the lifespan.
Memory, Sweet Memory
Regardless of storage medium, duplicate data will mean a much higher chance of being able to boot up successfully. A standard mirrored RAID can extend the lifespan of a drive (assuming it works) by many years. The more duplicates, the higher the chance the data will survive intact, possibly even doubling or tripling the lifespan.
- Modern (magnetic) hard drives use tiny magnetic fields on platters. Given the right circumstances, an unpowered magnetic hard drive will last between 2-20 years. At the low end of that range, the drive itself would still function, but the data may be corrupt; by the the end of that range, the lubrication on the drive will have failed, and the motors won't work. Theoretically, given perfect circumstances, the information stored on a hard disk will last for decades, but the drive itself will fail long before that. If the future user is willing to completely dismantle the drive and lubricate the motors, I could see a drive lasting for an additional decade or two. Interestingly, it's very difficult to find any websites describing how long a hard drive would last, unplugged.
- Solid state drives would probably fail within a decade. A subsection of SSDs (RAM disks) use RAM with a battery to keep the data fresh; those would fail very quickly. Regardless, the data doesn't seem to last as long as magnetic hard drives, and because of drive fragmentation, it would be much harder to rebuild as well. On the bright side, the drive itself would remain usable much longer, even if the data stored on it were gone; I can't find any good sources, but it looks like an SSD could last over 30 years in the right conditions.
- Punch cards last a surprisingly long time. They may not hold much data, but a stack of cardstock punch cards kept in ideal circumstances could last thousands of years!
- Magnetic tape has been shown to be perfectly readable in excess of 30 years; as long as the tape itself is undamaged, there is a high chance that data will last many decades. Duplicate data could extend that two or three times.
- Floppy disks rarely last 10 years, though I have personal experience with a floppy that worked after almost 15 years - however, other disks in the same box didn't last longer than two or three years. The chances of any given floppy disk working after a decade is quite low.
- CDs were initially promised to last hundreds of years, but in reality many disks failed within 5 years; most websites I found list the lifespan of a 'cold storage' CD as about 10 years, though there is a chance that a CD may last longer, as long as the foil backing doesn't degrade. Given the option, I would stick with DVDs.
- DVDs, especially high-end, commercially manufactured DVDs, may last 100 years or more; the data is stored differently than CDs, which means that as long as the disk itself doesn't degrade, the data should last a very long time - some sources claim over a thousand years in ideal circumstances. I would expect average media to last about 100 years.
- BluRay and other very modern optical storage have similar manufacturer claims of 100-150 years; there isn't as much data on those, but I would guess it is on par with DVDs.
- Flash drives, SD cards, and other small, cheap media will likely lose data within 10 years. The better drives may have longer lifespans, but I wouldn't count on it.
- Older EEPROM chips should last up to 30 years before the bits decay; some chips are very susceptible to UV light, however, so any sunlight would drastically reduce the life of the chip.
- Data stored directly in transistors and diodes (or even wires/resistors) should last nigh-on forever, barring physical decay; instead of using magnetic or electrical charge, they use a physical connection to power or ground to give the 1s and 0s. Obviously, this data would be read only.
Hardware
- Ok, I lied. The first thing to fail is probably the fan on the processor, rather than the storage. For systems with a fan, anyway. The fan is designed to spin all day every day, and without that motion to distribute the lubrication, it will quickly stick. Luckily, most computers will work without a fan... for a little while, at least.
- An optical punch card reader could, theoretically, last for thousands of years. It has so few components, and is designed to work with such high tolerances, it may well outlast the computer itself.
- Floppy drives need to be thoroughly cleaned after less than a decade. Dust and mold will cover the reading head and reduce even the best floppy drive to a floppy-destroyer. Once clean, however, the drive should last for at least 50 years.
- Similarly, magnetic tape readers will need to be carefully cleaned. On the bright side, the tape itself would clean the heads eventually, at the cost of whatever was on that first few feet of tape. Many commercial readers/writers are kept in hermetically sealed cases, which would keep the drive clean. I would expect a tape reader to last for 50-100 years, as long as the components hold out. The motor would need to be lubricated.
- Optical drives, especially drives that are normally sealed, will fare much better; solid caps will keep the drive working for 100 years, at least, and probably longer. Unlike sensitive hard drives, the motor on a CD, DVD, or BluRay drive will grind away without lubrication, for a little while at least. If cleaned, I expect an optical reader to last roughly as long as its media - between 50 and 100 years.
- Capacitors in the power supply or on the motherboard will likely be the first point of failure there, at around 15 years. The hotter it gets, the lower the lifespan; however, capacitors are fairly easy to replace; as long as the user turning on the computer knows how to fix them, they won't have any effect on the lifespan of the computer. Solid-core capacitors would fix that problem; there are a number of commercially available motherboards with solid caps, however, there are usually not available for power supplies.
- The BIOS, as well as various other chips on the motherboard, may become corrupt over time; I would expect modern chips to survive for several decades, and older chips to last even longer. Much older motherboards may use ROMs that are much less prone to corruption, and still older devices would use wires and AND/OR gates as memory, reducing the chance of corruption to almost nothing.
Conclusion
If a modern desktop computer were kept out of direct contact with weather (sunlight, rain, etc.) in an average climate (say, in a garage or shed), I would expect it to last for between 5 and 10 years. If the same computer were buried in a storage container in the desert (controlled climate, no sunlight, no damp), I would guess that it would last up to 20 years. The first thing to fail would probably be the data storage.
However, a computer with an SSD could be reinstalled with handy DVD installation disks; some computer could even boot and run from the DVD itself. Without a hard drive to worry about, the same computer would live another 10-20 years.
An old, single-board computer with solid-core capacitors, bulky wiring, fewer components in general, and much higher tolerance for noise (due to much slower processor speeds) may last centuries; as long as the chips and wires themselves don't decay, the device should keep working for hundreds of years.
Even then, there are still simpler computers that would last even longer. The NES, for example, uses a very basic processor with easy-to-replace components, and boots from a cartridge using a die-cast circuit for data. As long as they clean the contacts on the cartridges, someone could be playing Duck Hunt on an NES a thousand years from now!