I think its obvious to anyone that the answer is "Yes, they will advance faster" but its extremely difficult to nail down why.
Awesome question by the way :)
Electrical potential energy represents the power to do work. The more easily one can access electrical potential energy, the more free work one can do. Take water desalination for example: high energy cost to perform so if you make the energy cost trivial you can support far larger populations in arid areas. Similarly the first electrical powerplants were actually built inside of factories for those factories exclusive use - and perhaps a few houses around them. By making energy generation trivial you supply power to people's homes (and universities) far sooner then they would gain this free work potential through coal - this means people have productive evening hours (and electrical stoves /fridges) far sooner and have more time for additional work and/or development and population booms would have been likely (though plagues tend to keep those sorts of thing in check).
Additionally, even if the technology of the 1800s was primitive, this was an age where human rights were near non-existent and mega-corporations created world wonders. Railroads were built across North America in the early 1800s, The Suez canal was completed in 1869 and the East India Company was arguably the richest empire to ever have existed - and they did that with coal powered ships, not fusion powered ships. The world would have been explored, connected and exploited far faster with fusion than it was with coal by a small group of rich megalomaniacs before the age of red tape and humane behavior hemmed them down. These things would accelerate tech.
Oddly enough, to the extent of my noggin to perceive, the actual potency of limitless power doesn't become relevant until fairly late in the technological cycle. You don't actually try use large amounts of electricity for things like the large hadron collider until pretty late into technological tree (and there is a limit to how fast you can make a wooden ship go with a fusion reactor before it rips apart :). Rather, it is the cheapness of producing electricity for low consumption appliances which grants a thousand small boosts to societal efficiency which really gets the tech barrel rolling faster.
PS. For the handwavium source of this endless power, Nikolas Tesla thought it would be possible to beam electricity into the stratosphere and beam it down anywhere around the globe. He did not know at the time that the stratosphere already has absurd electrical potential without being topped up. Just make the claim that humanity found a way to draw electrical potential from the stratosphere - heck, just say that the Egyptian pyramids did it naturally and the rest of the world just copied the design without understanding it :p