The purpose of combat is not killing but control. Terrestrial combat uses lethal force because it is the most effective way to take control of a battlefield. But if the enemy surrenders, there's no more need for killing. In fact it's better to be merciful with those who surrender so the rest have less incentive to fight.
Everything in space is counter-intuitive. Shooting a bullet through the hull of a space station is not a big problem. Air only leaks through at a limited rate and the hole can be plugged with a wad of bubblegum. If the hull is hardened against micro-meteorites, the bullet won't even penetrate. The biggest problem with using guns in microgravity is the recoil, which can easily cause any soldier to totally lose control of their orientation and position.
Swords aren't much better. If you swing a sword, the rest of your body will turn the other way. Using a sword would be extremely difficult. A spear might be a tiny bit better because there's not as much rotation involved (though you'd be surprised), but your reach won't be great because you will move further from the target as you trust the spear. Microgravity combat would be largely concerned with securing good anchor points in the environment to enable any kind of effective action without obnoxious physical reactions.
A defensive party that's prepared to be boarded will get themselves strapped in to fixed positions and get ready to fire guns or trust spears. A boarding party will be in free-fall when they enter, which is a huge disadvantage. The defending crew has control of the interior environment. But the boarding force has control of the external environment. The boarding ship can try to knock the defenders unconscious by subjecting them to high accelerations - probably by spinning the ship they are assaulting. This kind of action could kill everyone on board, or force them to detach from their anchors.
The simple fact is, however, that boarding parties in space are probably unnecessary. Having control of the exterior of a ship allows a siege to be completely effective. Cover the target vessel's radiators with reflective insulation, block their sublimators and cook them out. Life in space is so tenuous that it's hard to justify fighting back - or fighting at all. (This is why space colonization is such a great goal for peace.)
Now if you really, really want to have boarding parties in your story, try this tactic: You can't fight what you can't see. Inject a spray of black, low viscosity, highly wetting liquid that will coat the optics of anyone or anything inside. In microgravity, fluid dynamics are dominated by surface tension effects. A blob of liquid touching a surface will tend to completely coat the surface, unless that surface is phobic to that particular liquid. Most if not all materials currently used for optics (glass, plastics, the human eye) are at least somewhat hydrophilic, so a spray of blackened water would easily coat and interfere with these. (Just ask astronaut Chris Hadfield how bad it can be to get something in your eye in space.)
Edit:
If your combat environment is subject to spun gravity, it's important to note that projectile weapons will behave differently than when subject to normal gravity because of the coriolis effect.
Let's model the habitat as a cylinder spinning on its axis. Use the right hand rule to define North as the direction you would have to face to see the cylinder as rotating clockwise - or if you point the thumb of your right hand in that direction your fingers bend in the direction of rotation. South is the opposite direction. West is defined as the clockwise direction. East is opposite West.
If you stand on the inner surface of this cylinder and fire a projectile, it's path (as seen by you in the rotating frame of reference) will be significantly different depending on what direction you fire.
If you fire directly North, the bullet will initially go straight, but will start curving to your right as it "falls" to the "ground".
If you fire South, the bullet will fall to your left.
If you fire East, the bullet will fly straight, but will fall slower than you'd expect.
If you fire West, the bullet will fly straight, but fall quicker than you'd expect.
If you fire straight up, the bullet will go straight up, then curve West, then down, and then maybe start heading down and East - it really depends on muzzle velocity and the spin rate of the habitat. The trajectory that you see could end up looking kind of like a "Run for the Cure" ribbon. If you get the muzzle velocity just right, you could get the bullet to come back and hit you - but it would hit you from up and to the West.
If you fire straight down, the bullet will curve East and probably hit the ground.
Fire up and a bit East to hit a target on the exact opposite side of the cylinder from you.
If you try to drop an anvil on a roadrunner from high "up", it could take a very long time to get to the big red target you've painted on the ground. The time it takes depends on the diameter of the habitat and how high up you are. If you were fairly high "up" - close to the center line of the cylinder, but not quite at it - the anvil might go all the way East, come back around a few times, apparently picking up speed in the Easterly direction until it finally creams your target going practically sideways.
You can combine these for trick shots, but there's also the complication of air resistance.
The coriolis effect becomes less obvious as the radius of the cylinder gets larger, assuming that the angular velocity of the cylinder is chosen to provide one gee acceleration to a person on the surface. A gargantuan habitat may only rotate several times a day, but a small habitat would have to spin up to several times a minute to yield one gee at the surface. So smaller habitats will have far more pronounced coriolis effects than large habitats. (Earth rotates once per day, so we don't really notice the coriolis effect in our daily lives, but it's there.)
Another neat feature of spun habitats is that you can tell which way you're facing by turning your head. If you face North, then turn South, you'll have the unpleasant sensation that you are spinning counterclockwise - at twice the rate you are actually spinning counterclockwise - and you might tip over if you try to walk too soon after turning around. (You can simulate this by looking up and spinning around until dizzy, then stop and try to walk around without losing your lunch.) Spinning habitats are not much fun for this reason and that's partly why the ISS has no artificial gravity section. It would take a seriously large artificial gravity ring to provide an environment that doesn't feel like a carnival ride.
weapon
tag on Space Exploration SE, and perhaps in particular Have spaceships been tested to withstand bullets? That even references back to our question How to solve the old 'gun on a spaceship' problem? which has plenty of answers. $\endgroup$ – user Apr 21 '17 at 14:53