And more importantly, how could we not know about it? I remember hearing that most of our man-made monuments and buildings will have completely collapsed in a few thousand years, so is it possible for a previous intelligent civilization to have existed in the past but have completely faded away by the modern era?
No, more advanced human civilizations could not have existed and us not know about it. The mark that a civilization on our scale makes is a very big impact on the world around us so much so that it would be fairly obvious that any culture any where near our level of technological or global reach would be severely obvious. Despite what nat-geo and the history channel say about our buildings falling apart and such. Even if all that were left were the natural world and all the rest had vanished our resource extraction and farming practice remnants would make it obvious.
There could be civilizations that we have not found that have raised up beyond the level of what we commonly expect for the age that it was around in though. The reason we wouldn't have found them is because we destroyed the artifacts accidentally, there are sanctions on looking in those regions of the world, or they are very dangerous to look into those regions.
Also we have a pretty solid understanding of how our species spread, when and where and we have a general outline of history from the point from which it were possible to start having civs on Earth by humans to present so it is almost impossible for an advanced civ to have been on Earth without our knowledge.
That being said, if you're a super advanced alien race you could probably cover your tracks, however any ancient alien myth completely throws this idea out because ancient aliens either came to help and thus it wouldn't make sense to cover themselves up or were tossed out which means they simply couldn't cover it all up.
Edit: I want to add that if a race wanted to cover up that they were here so that it would be really hard to figure out it would take a very long time to do, literally millenia if not millions of years and complete resurfacing of the planet to the degree that it's really not a reasonable thing for such civ to do as it implies they'd be extremely hostile.
If they were humans, we would know. We can be pretty sure of this because humans/hominids have only existed for a few hundred thousand years, which isn't long enough for evidence of settlements to completely erode.
However, go back far enough, and it's absolutely possible. After a few million years, for instance, the only real remnants of a civilization would be in the fossil record. If you find a disproportionately huge number of fossils of one species, spread all over the world, this could serve as inconclusive evidence of an intelligent species, as the only other species that has done this is man.
The best form of evidence for an earlier civilization would be the presence of man made elements. If, for instance, levels of naturally occurring plutonium were much higher, that would indicate the existence of intelligent beings in the past who had nuclear technology.
That depends on how far you are willing to go back. An intelligent dinosaur civilization at the level of the Roman empire would probably not be detectable today (however assuming our theories about how biological intelligence works are right, we would have to find dinosaurs with large brains — but then, maybe we simply didn't yet look at the right places). On the other hand, a humanoid civilization of 20th century level one million of years ago would almost certainly have left traces in the geological/paleontological records that we'd be able to see today, even if we wouldn't find archaeological artefacts (but at that level, I'd expect to find those, too; e.g. a DVD may not be readable in a million years, but I'd guess it would still be recognizable as an artificial object).
We can examine geographical features and discover the early history of the earth based on traces of minerals in rock layers (such as the iron oxide deposits caused by the "oxygen catastrophe"), or microscopic fossils of bacterium 3 billion years old, so the sorts of disturbances created by civilizations thousands or millions of years in the past should still be apparent.
Anomalous deposits or layers in the geographical strata, unusual concentrations of isotopes and other markers would generate a "WTF?" response among geologists, and eventually filter through the scientific community to seek to understand the meaning of these markers.
The idea that more recent civilizations may have existed during the last ice age (a rather popular trope in the "alt history" community) and was swept away in a global flood 12,000 years ago when the glaciers melted does not even pass the sniff test; archeologists can reliably find, date and understand human artifacts from the "Cultural explosion" some 20,000 years ago, and we know anatomically modern humans have been around for 100,000 years. If the Ancestors had an advanced civilization 12,000 years ago, the evidence would be totally unambiguous. There is also the small matter of what happened in the 8000+ years between the end of the putative Ice Age civilization and the start of recorded history? If people were smart enough to build an advanced civilization 12,000 years ago, would they take a siesta for 8 millennia afterwards?
An extremely advanced civilization could possibly advance beyond metal and plastics fully into bio-engineering, creating organic cities that grow and repair themselves and live in a perfect balance with nature. Such a civilization could decide that leaving monuments and permanent structures are like scars on nature, and actively remove all traces of their industrial origins themselves, and focus on zero-impact harmony with the biosphere as a civilization goal.
We are surrounded by such amazing complex biological technology that we scarcely understand it at all (and currently suppose it appeared from slime, by random and chance). IMHO its plausible that the current biodiversity and ecosystems ARE the evidence of previous civilizations. Our chromosomes and folded protein structures could be the archaeological items left over from the earlier civilizations engineering work. Who knows where they came from, that's another question :p
As to mount Rushmore, its suggested lifetime of half a million years is still only 0.02% of the planets age.
What I know (based on mass media) is WE DON'T KNOW
According to Ancient Astronaut Theorists, we did, for some of the mega structures of the ancient times are impossible to be built without the mechanical and engineering capabilities we have today. That there are large stone monuments (not Stonehenge) that are precisely cut and placed on point of each other, so much for rock and chisels that we know they have back then.
But scientists and engineers are finding solutions on how ancient civilization developed these structures. Like how ancient Egypt built the pyramids.
Still, it remains the same, We don't know. What we do know would be based on our knowledge, and how we could prove something. But their mere existence is a big question. We stand on, we believe on what we see. Most of the times, when some people see immediately they believe in it, no more research on the subject matter. That's why some people fall on false information on Facebook. If we haven't seen it, we won't accept it. Well, except if you have faith in it, but that's another matter.
I like the points made by the above answers and want to point out another; Mount Rushmore.
Mount Rush erodes at a rate of 1 inch every 10 000 years, with each of the noses being about 240 inches long, so they would take some 2.4 million years to erode, barring an ice age that sent glaciers down to scrub them away.
Of course, the faces would start to lose some definition after half a million years or so, but they'd still be recognizable as faces.
If there was an ancient civilization as advanced as us or more, they would have monuments and these monuments will last.