the moon would not move in the sky and there would be no tides
Wrong. The sun provides tides too (albeit smaller ones than our moon), so the sea still rises and falls on a regular cycle. You'll even get variations in tide height depending on the relative angles of the sun and your "moon", though not to the same extent as spring tides on earth. This means that it may still be possible to use tides on your planet to calculate the mass of the moon in the way that was done historically here (see the quora answer linked by AlexP above) though as I am not a mathematician I couldn't tell you the exact technique or how precise it may be.
Establish the distance to the "moon". Our moon's distance can be judged using lunar parallax. You'll have to observe your moon from two different places on the surface to see the effect, of course. This'll also get you the size of the moon, so you can see that it is about the same size as your own planet, another big clue.
Determine where the barycenter of your "moon"-planet system lies. I believe you can do this using diurnal parallax, only in this case your planet doesn't rotate itself, but it does rotate around the two bodys' barycenter which will cause visible parallax effects. Not big enough to see the stars wobble, I suspect, but it should be enough to cause other planets in the same solar system to show some movement.
Now you've determined that the barycenter of your system lies at more-or-less the midpoint of the two worlds, that's a massive clue that they mass about the same, assuming you've got some idea of the nature of gravity.
3. Tidal locking
There's a big clue in the tidal locking, of course... I'm not certain on the history of understanding of tidal forces on solid a objects, or how the idea came about, but you'd still get tides on your world so it wouldn't require a huge leap of logic. If tidal forces were understood, it may well be seen that bodies will become tidally locked if they have similar masses, or if they have existed long enough to lose their rotation to tidal effects.
First, determine the mass of your planet (there are various ways to do this, see this earth science SE answer for example, but I won't go into them here). Next, establish the age of your planet (again, left as an exercise to the reader). It should be possible to determine that insufficient time has passed for your planet to become locked to a much less massive body, therefore the mass of your moon must be pretty similar to the mass of your planet.
I think. There are a lot more leaps of logic here, but it wouldn't do to underestimate people's ability to figure stuff out!
4ish. Meteorite Impacts
Meteorites will sometimes hit your moon. They sometimes hit the earth. There's good reason to believe the meteorites may be made of the same material. Given that you know the size of the moon and how far away it is (see parallax above), it is possible to estimate the height of the craters. Assuming your "moon" isn't made of something particularly bizarre and scifi-y, it'll be possible to make estimates of the density of its surface given the sizes of the craters and the spread of ejecta, given some understanding of gravity and ballistics.
I'm not totally certain that this will provide a useful estimate of the mass of your moon (because you have to make assumptions about density of subsurface layers), but it is certainly a contributing piece of evidence, especially when combined with assumptions about a common origin of the "moon" and the observer's planet.
(note that this does make some assumptions about the nature of the "moon". One with a thick atmosphere, or a liquid surface or whatever else may make this bit of analysis impossible)
(note 2: there's a related useful thing here if the "moon" is a rocky world, because you can use impact cratering to help estimate its age, which is a pointer towards a common age and common origin of the observer's planet and the "moon")