You actually have two separate issues: closing the outlet from the reservoir and then draining all the pipes in the city.
Shutting the municipal water supply might not be as difficult as you think. If you can find the city engineer's operations centre, it may be possible to close the valves from that point and stop water from flowing into the city water system. Based on the OP, there is still electrical power so the computers and vales should still work.
Draining a city sized system is going to be more difficult. You can theoretically drain part of the system by dumping the municipal sewage system and essentially creating an "open" system between the inlet and outflow, but the problem is the water trapped in millions of pipes between the reservoir and the outlet. For that you would literally need to go from house to house and building to building and open the taps so air can get ion the system and the water has a way to flow out (the air can get into the taps, and the water drain back down to the inlet manifolds, I'm not talking about draining out through the sink). Larger building might have valving to allow the water system to be closed off and drained from the utility room, but this largely depends on the building, when it was built and what systems were installed. You might have to turn off and empty the water heaters in order for the water in the pipes to flow out.
This could take quite some time. XKCD once did a comic asking how long would it take to visit every room in NYC, with the answer being @ 10 years. NYC is a very large city, so unless you are thinking about a megacity like New Deli, Tokyo, Shanghai, Mexico City etc. you could conceivably drain the water out of most buildings in a year or so.
Incidentally, this really is only an issue if the temperature is below 0C for several days in a row, when the water sitting in pipes freezes and expands. If you are really all alone in the world, it may be much simpler to head south and find a place with is not prone to sub 0 temperatures, in which case your major issue will be spoilage of foodstuffs and organic materials.