Boiling water, ice cold water and a binary search
You did say crude would do...
Humans can't do absolute temperature measurement, but they can do relative - I can't tell you how hot my mug of tea is, but I know it's hotter than my hand and cooler than the kettle. This means that if we have a range of objects whose temperatures we know, we can tell if the thing we want to measure is hotter or cooler than them, and so get a range of possible temperatures - with some reservations.
This is where the boiling and ice cold water come in. Let's assume the ice cold water is about zero degrees on an arbitrary scale of temperature. (we can't use solid ice because of latent heat, as AlexP pointed out), but we can use water that's just above freezing. Let's then assume the boiling water is 100 degrees on the same arbitrary scale. If we mix equal quantities of boiling water and ice cold water, we should get a liquid that is 50 degrees on our arbitrary scale.
Now check the temperature of your water against this liquid. Is it warmer or cooler? If it's cooler then your water is somewhere between 0 and 50 degrees.
Repeat this process again with a 3:1 mixture of ice cold:boiling water, and keep adjusting the fractions until you've narrowed it down enough.
This only works between 0° and 100°C at best. In fact, because humans burn easily, it only works well below about 45°C. Helpfully this is about the range specified.
Another issue is thermal conductivity - if two objects have different conductivities they will feel different even if they're the same temperature. Fortunately once again, we're comparing water to water so their thermal conductivities are the same.
It also suffers from the fact that the ice cold water will be slightly warmer than 0°C, that the mixture will cool fairly rapidly from its nominal value, so your measurements will be less than exact, and that ice isn't always easy to come by.
Other than that, this method should be fairly accessible and uses practically bronze age technology.