So context for the industrial era: transportation and movement of information were one of the biggest revolutions in technology and society in the 19th century. With the invention of the steam locomotive, people and goods could be moved across continents in as little as a week when it would take months prior. People could now move to cities where new jobs had arisen. Goods from far markets could be transported, bringing new variety into people's diets.
And the telegraph had an immense effect as well. You could now instantly transmit information across the country. News stories from foreign lands could now appear in the paper mere weeks after they'd happened. Generals in the US Civil War could coordinate large movements of armies, etc.
And that's where we get to warfare. The US Civil War is a very good example of how war was changing in the industrial age, and WWI, with mass troop movement by train, allowed for killing on an unprecedented scale.
The limitations of your system however, in terms of size and weight, will alter things somewhat. You place your story in the early industrial - so 1750 to around 1830s I assume. Movement of large cargo in war, like artillery, supplies, and troops, would likely be moved aside to be moved overland, by canal, or by early railways - and perhaps your magic system would cause railways to never be used - I find that unlikely however due to the vast benefits of railways in pulling heavy loads.
Anyway, the big benefit I see for your teleportation magic in war is in information. Even if the telegraph is yet to be invented, if this is based on weight, as it seems to be, you could teleport a number of letters instantly. While it would require infrastructure as you mentioned, it would have the great benefit of that anyone could receive such messages - with telegraphs, you required a trained operator who could use the machines and knew morse code.
Not only would this be useful in warfare but it would become extremely profitable to relay information throughout your world. It might even pose a serious threat to the invention of telephones later. I don't see telegraphs ever existing.
In tandem with information is espionage. The ability to teleport would be very valuable. I'm not certain how your description works exactly, but if a spy could be teleported quickly to an enemy camp, get some war plans, and get out quickly, it would change EVERYTHING. There would be copious fake plans, intense scrutiny with guards and such, because you never know who could jump in, pretending to be a soldier, and immediately steal something important.
I mentioned earlier that weight might get in the way of moving things like artillery and troops. However, this goes out the window if you want speed. You trying to get around the back of the enemy camp in the middle of the night but a river is in the way? Quickly transport a portable bridge through a portal (and bridges were an annoyance in the US Civil War).
The weight/size restriction on your teleportation would probably be the bigger restriction in warfare. You might hold back initially on the cost, but then when the other side gambles in burning a bunch of money to get some troops somewhere fast, you could lose the war. To this end, you could see intense guerilla tactics - like the espionage, this could be one of the worst annoyances of this kind of war.
In summation, this would have a dramatic effect on war in the period - especially on speed, and you'd get to a point where this could be a big cause of war debts from burning money to get troops where you want.
One very important consideration however, is if the teleportation is new, or older. This will change the entire way you depict the war. Maybe teleportation was used in past conflicts, but not to this extent. Maybe it was taboo. Maybe it was considered unseemly to be used by common people. If you want a war where everyone is un-used to the applications of this tech, that's what you're going to want - and the theme of everything being new and unfamiliar shares commonality with other things in the period.
If teleportation has been used extensively in past wars, things like its uses for communication and espionage will be much more understood, and trained for. However, with the onward march of technology, especially in the era, people will inevitably find unique new ways to capitalize on their tools to outmaneuver their opponent.