It depends on how you define precision, and also whether you place more importance on what would be commercially viable technology for mass production in the Victorian era, or what would be the best cutting edge fabrication processes available to the era's top scientists and instrumentmakers. For example, would the best precision of our era be represented by computer chips, or by LIGO?
In the latter case, the answer would probably be "remarkably good precision".
I think the best I can do is to cherry pick the era's most advanced instruments to serve as examples. So:
- Diffraction grating ruling engines.
These are a classic example of machines that push the limits of mechanical precision, and they also directly match your definition of "move a tool in a controlled and measurable fashion, e.g. a saw that lets me cut a groove x wide and y deep". By the end of the Victorian era, Henry Joseph Grayson had constructed ruling engines that could scribe gratings with 4700 lines per mm (ie, 212 nm pitch; each groove can roughly be assumed to be about 100 nm deep). Previous ruling engines were already pretty good, too.
- Michelson interferometer
The Michelson-Morley experiment was conducted in the 1880s, and represents one of the first and most important applications of the interferometer, which is still among the most sensitive measuring instruments in existence today. Their white-light interferometer was built on a sandstone block floating on mercury, which would compare well with setups in today's top academic labs (for precision if not for safety). Although they were looking for optical path length changes due to aether wind with fixed mirrors, their interferometer would have been sensitive enough to detect the mirrors moving by as little as 2-3 nm.
- Whitworth's comparator
Accurate micrometers for use in machine shops were already being commonly made in the Victorian era, but undoubtedly the king of this category of instrument would be Whitworth's comparator, built in 1871 and which could measure differences in the lengths of objects to submicron accuracy.
- Chemical etching
The smallest scale parts might be made by chemical rather than purely machining techniques. For example, platinum wire of about 1.5 micron diameter, known as Wollaston wire, was produced by the early 19th century. It is first embedded in silver, then drawn down, then the silver is chemically dissolved. If you wanted to make gears the size of specks of dust in the Victorian era, you could try to do it this way. (By the way, microscopic clockwork will make your clocks smaller, but probably not better at keeping time...)
- Coating processes
Although this isn't directly motion control related, it's also relevant to fabrication abilities. Electroplating became a mature technology during the Victorian era, and other thin-film deposition techniques such as vacuum sputtering were invented during that period. Electroforming was invented by 1840. These additive-manufacturing techniques allow the creation of extremely thin and uniform films and delicate thin-walled structures. A coating of silver on a glass substrate could be made less than a micron thick. Electroformed thin-walled metal objects could be tens of microns thick.
- Bonus: Division of the circle
In addition to length, another important thing is accuracy of angles. For example, if you have gear teeth, how evenly-spaced are they. The Victorians would have had little difficulty here. Already in the pre-Victorian 18th century, Jesse Ramsden had built division engines accurate to one arc-second.