Serious scholars of mathematics tend to be educated in the history of mathematics and mathematical notation as well. As such, the question is complicated by the fact that the time traveling mathematician is not entirely helpless, he has advance knowledge.
As far back as the Renaissance, there would be hardly any difficulty. They pretty much did math like us. For example, Descartes was one of the originators of denoting variables with $x$ and $y$. Of course, if you are interested specifically in discipline X, and you arrive just when X is being invented, things will be in flux, and there will be multiple competing notations that confuse matters somewhat. Calculus is the classical example: Leibniz and Newton both independently discovered it; Newton favored $\dot x$ and $\ddot x$, while Leibniz used $dx/dy$ and $d^2 x/ dy^2$. I don't recall now how $f'(x)$ and $f''(x)$ fits into this.
Calculus is an interesting exmaple for another reason, though: Typically common calculus textbooks will include this little story about the discovery of calculus, and many professors enjoy retelling the tale when teaching Calculus I. This nicely illustrates my point about mathematicians having a head start due to their knowledge of math history.
In Middle Ages things get slightly funky: This is when Arabic numerals were brought to Europe (in India and Arabia they had been in use for centuries), so before that time, things might start out a bit confusing to a modern mathematician. However, the alternatives like Roman numerals or Mayan numbers are really not complicated (I personally find Mayan numbers more elegant than ours), and anyone who is interested in math can easily learn them in a few hours, if they don't know them already. Mathematical thinking also seems comprehensible, going by examples like Fibonacci.
In Dark Ages Europe it may be difficult to discuss math due to the odd attitudes people may have towards learning in general, but assuming you do find a cooperative scholar, the math itself should be easy. Of course, concepts such as variables or equations (as we know them) had not been established yet, so discussion may be a bit cumbersome ("thus we find that number which is thrice that number of which the square is a tenth of the number which...") until the interlocutors have been "enlightened" with modern algebra.
I'm not sure what state Roman mathematics was in, but surely they would be intimately familiar with the heritage they received from the Greeks. Discussing math with the Greeks might be positively pleasant: The Ancient Greeks pioneered so much of our philosophy, logic, math and geometry, and their writings show an exceptional clarity of reason. Not having knowledge of modern notation, perhaps the way in which they talk about math would be a bit strange, but the essence of their thoughts should be familiar, given that to this day we have students of mathematics retrace their footsteps when learning. Incidentally, Greek mathematics is very readable today. I believe Euclid's elements was commonly used as a textbook of geometry up in many places up until a couple decades ago.
Further back, there are some mathematical manuals from Egypt that give a fascinating glimpse of truly ancient mathematics. The Rhind papyrus appears to be some kind of math textbook, complete with example problems and reference tables, very similar to textbooks we use today. Going through the problems, many of them are very obviously similar to elementary math we learn in school: Calculations of volume, work, fractions, and so forth. There is also a great example of where the difficulty might be: Egyptians, apparently, liked to represent non-whole numbers as sums of fractions with quotient 1, eg. $3+1/6+1/7+1/13$. This bizarre notation can lead to some very confusing numbers (they got confused too, and they made tables of fractions to cope with it) for a modern reader. Their units are also quite strange. But once you start systematically writing down the fractions and equation in a neat manner, as should be instinctive after a modern education in math, everything is very clear.
There seems to have been a similar situation in ancient China. Taking as example their approach to calculating areas and circumferences of circles we can see where the biggest issues might be: These people had not yet standardised $\pi$, nor did they fully understand its nature, so familiar formulas like $\pi r^2$ become $730/232 \cdot r \cdot r$, or $2\cdot \sqrt 10 \cdot r$. At first, these weird constants might throw you off (they are approximations of $\pi$ that haven't been needed for centuries) but it's not something that anyone educated in math would not know, or couldn't figure out. I would guess that in other cultures, such as ancient Amerindians, the math would likewise be straightforward besides a few trivial oddities.
In general, math is different from language. Unlike languages, all human mathematics appears to converge to some universal (or possibly metaphysical) truth -- it is no wonder the Greeks venerated it religiously. Whereas language is for the most part arbitrary and varies greatly between cultures, math is convergent: "Great minds think alike". Mathematicians are in addition taught how to analyse and dissect problems into math - while typically people are not taught research linguistics in school. So I would say that besides communication itself, and strange archaic notations, math from any time period past would not pose any problems to a modern, educated person besides obviously the issue of them not having discovered certain useful things yet (but those can easily be taught). It is interesting that in Contact, the aliens begin their message with elementary math, as this ensures that it will be readily comprehensible. It is a very relevant comment on the relation of humanity with mathematics.
Even the math of the near future would likewise be manageable; but once you go thousands of years into the future, it may turn out that a massive revolution in math has completely changed the way people think about it (not to mention possible impact of ubiquitous computation technology and AI). By the way, historically mental/manual arithmetic was a vital skill for many mathematicians, but modern scholars are blissfully freed of this burden by calculators. A time traveler may have trouble being taken seriously at first, due to his poor calculation and rote memorization skills, but this barrier should be easy to overcome once the merit of his knowledge is demonstrated.
