Leopold Kronecker is quoted as saying "God made the integers, all else is the work of man." In the original German, integers are called natural numbers. Your aliens might replace that with "God made Gaussian integers," but they will still study ordinary integers.
When you get down to it, mathematics is the science of evaluating models that are based based on axioms and drawings conclusions whichare more (or less) relevant to real life. If you learn mathematics rather than primary school arithmetic, you usually start with extremely simple axioms and models and then progress to more complicated ones. And when it comes to applied mathematics, you should always try if the simplest models give interesting results before you try to fit a more complicated model to the facts and problems. That applies in calculus, geometry, set theory, graph theory, statistics, you name it.
The concept of a prime matters when one does prime factorization, for instance, and that in turn is useful for cryptography. Not considering 2 a prime number does not change the fact that such numbers are useful -- you would have to invent a new name for the property of integers being composed that way. Or take group theory.