Maximum railgun and coilgun dimensions for space craft
This isn't quite about maximum gun dimensions; it is more about post-justifying your existing design. You've already decided that 180km long warships are a thing, for example, and that'll be subject to all sorts of interesting forces and construction issues too. There's also a critical difference between "Is my solution to this problem justifiable?" and "What is the best way to solve this problem?" and I can't help but feel if you'd ask the second question you'd get a lot more answers, and they'd be a lot more interesting, too.
Anyhoo, enough with the grumpy nitpicking. And apologies for the wall-o-text... this grew a bit in the answering :-( There's a suggested alternative weapon in the edit history, but there's too much scroll here already so I removed it.
TL;DR: your weapon is implausible and magic, and as such you can hand wave it to be literally anything you like.
Is there a maximum rail length for a rail gun/canon until they become ineffective?
As length of anything increases too far you get issues with stiffness. Incredibly stiff materials at the metre-scale will behave like wiggly noodles at the hundred-metre-scale. Railgun rails are subjected to intense bending and warping forces, so a very long railgun such as yours might have serious issues remaining smooth and straight... and failure to do so when firing a relativistic projectile is going to be Very Very Bad for your planet killer.
Moreover, the rails will be subjected to intense heating from the plasma arc that forms behind the projectile. Given the power of your gun, this arc is going to resemble a coronal mass ejection. Good luck keeping your rails and indeed ship intact!
Would a value of 100km suffice? The projectile is a cylinder that is 1km in diameter and 2km in length. At this length, would a coilgun be better to accelerate the projectile to the required speed?
Given your requested muzzle velocity, you're asking for accelerations of the order of 2.5x1011m/s2... that's like the surface gravity of a neutron star.
I realise that prototype coilguns have managed accelerations a little bit like this (albeit a few orders of magnitude lower) over short distances, but realise that these were very small projectiles being accelerated for extremely short periods of time, not guns shooting relativistic asteroids!
So, to answer your specific question, a coilgun would be better than a railgun. But really, what you're asking for here is magic. It is made of pure handwavium... you've even admitted as much with your power supply. As such, I would simply not describe how the gun works at all, or even the principle it uses. It is so far beyond our own technology that no-one can know what such a device would look like or how it would work. Conserve your detail for where it matters!
Would an exit velocity of 70%-80% of the speed of light make sense?
Lets imagine a low-density projectile, say ~1000kg/m3, so its a bit like ice. It masses about 1.56x1012kg, and there should be plenty of icy asteroids or moons available to get the matter from.
The Lorentz factor at 75% of lightspeed is ~1.5, so we can't neglect relativistic effects. Breaking out the relativistic kinetic energy equation, we can see that your projectile has a kinetic energy of ~7.23x1028J. This is less than the gravitational binding energy of the Earth (~2.4x1032J) but pretty close to that of the moon (1.2x1029J) so it is at the upper limit of muzzle velocities if you don't want to reduce smaller planets to asteroid belts.
You did ask for some numbers and formulas, so here are some.
Luke Campbells approximation for coilgun performance models a coilgun as a device that converts the magnetic field energy in a gun barrel to kinetic energy. Given a kinetic energy of ~7.23x1028J, and the volume swept out by your projectile in the barrel, you can see you need an energy energy density of ~9.2x1017J/m3 in order to deliver enough energy to your projectile. The formula for the energy density of a magnetic field $u = B^2/\mu_0$, where $B$ is the field strength and $\mu_0$ is the permeability of free space. Feeding therequired energy density back into that will show that you need a magnetic field strength of about 1.08x106 tesla.
This is the sort of field strength associated with the surface of a neutron star, and will be Quite Hazardous for anything made of regular matter. It will probably wreck atomic bonds and crystal structures. It is also far higher than the critical magnetic field strength of any plausible superconductor, meaning you can't use superconducting electromagnets to drive your coilgun, and non-superconducting electromagnets are going to have to dissipate solar-flare levels of energy when your gun fires, probably blowing your ship to pieces. I won't go into the hoop stresses exerted on your magnets, or consider the vast mass of reinforcement the gun would needed to avoid bursting as soon as the magnets were turned on, but rest assured this is yet another near-insurmountable problem!
As Knowads mentioned above, the force exerted by a railgun on its armature is described by $F = I\ell B$ where $I$ is the current flowing through the armature, $\ell$ is the length of the rails and $B$ is the strength of the magnetic field.
Given a required acceleration of 2.5x1011m/s2, the ice cylinder projectile needs a force of ~3.97x1023N applied to it (which needs to be applied carefully... if you just tried to push the projectile you'd exert a pressure of >5x1017 pascals which is so many orders of magnitude higher than any plausible molecular binding strength you'd smash the projectile to dust in an instant). Anyway, feeding that force into the force-onna-wire equation gives you ~3.97x1018 amp-teslas. That means you're going to need at least a current of 40MA if you're able to arrange a magnetar-strength magnetic field (and read the description of what that does to normal atoms!). For a weaker field, like one associated with a regular neutron star, you need tera-amps of current, and google isn't turning up any real-world uses of such a ridiculous unit for me.
