As mentioned in comments, I don't have the full stability answer to hand (although see edit below). But I do have a practical answer.
The practical answer is that no feasible human effort could change the orbit of the moon via Earth-mass-transfer.
Here is how the net attraction between the earth and the moon would change as you moved Earth mass to the moon (Moon is approx 1.2% Earth mass), using Newtonian gravity
Wow! Peak 12 times increase in gravitational force?! Okay, sure. I'm still not sure that the change in barycentric rotation won't perfectly compensate for that to leave us in a stable orbit but yeah, there's a big potential change in net attraction.
Let's zoom in on the left of the chart:
See that red marker? Waaaaaay down there on the left? That represents moving the ENTIRETY OF THE OCEANS TO THE MOON. It changes the net gravitational attraction by 0.01%.
Therefore, even if mass transfer could destabilise orbits (and I am not sure that is true), the amount of mass transfer that could be achieved is incredibly insignificant to orbital dynamics.
EDIT: Having thought about the general stability of mass transfer from the moon to the Earth, I think the question isn't properly defined. To define stability we need to know both the change of mass and the change of velocity of the Earth and the Moon during the mass transfer. Keeping constant velocity sounds like the simplest option, but it is clearly unphysical: imagine we transfer mass such that the Earth and Moon are the same mass; they now have a barycentre [the point about which they rotate] exactly halfway between them --- meaning both must have equivalent tangental velocity around this centre. But we have also assumed that the Earth's and Moon's tangental velocities don't change. Therefore we have a contradiction. We can only solve this by making explicit assumptions about momentum transfer.
EDIT2: A practical consideration I forgot to mention previously. Gravity is a harsh mistress that makes moving mass very expensive. If you wanted to build extensively on the moon it would rapidly become preferable to mine materials from the asteroid belt (send a small set of mining materials up-gravity, send lots of mined mass down-gravity) rather than sending up a full set of mass from Earth to the Moon. See https://xkcd.com/681/ for a rough idea of the Gravity Well energy problem