How would melee combat develop in a zero-g environment? What kind of formations, and techniques would be developed to combat both other zero-g melee combatants and arrow fire?
This applies for both man-to-man combat, zone defense combat, and formation to formation combat.
Your primary concern in a zero-g melee fight is going to be Leverage. Any kind of melee strike needs leverage to drive home any sort of impact, be it a punch, kick, stab, slice, or anything. I can think of two major possibilities.
The first is any easy reliance on handholds and footholds in the corridors. If the environment is meant for transit, like a hallway, there will likely be small bars or nets or the like to allow people to easily move about the ship or station. In combat these could be used to anchor the combatants. Like in Ender's Game, the battles would be quite disorienting to a person used to 2D engagements since the person you're fighting might be anchored to the ceiling, but these would be far more reasonable than the second option.
The only other place to gain any leverage would be on another person. This person could be friend or foe, so grappling will likely be the dominant form of combat in this scenario. Launch yourself at the enemy, grab them, and wrestle until someone wins. Advanced techniques might include complicated multi-man chains to allow easy scrambling from one foe to the next, and tag team wrestling of sorts where numbers can be used to overwhelm enemies. Formations of combatants would have them all latched on to one another like skydivers, allowing for wrestling to take place while still keeping the foes from breaking through the formation. It could get crazy.
Personally, from a story perspective, the multi-man grapple brawls sound like an excellent story hook. In reality, most ships or stations will probably have handholds all over the corridors to allow for easy movement, which should make combat more familiar.
The thing with zero-g is that nobody realizes just how much we rely on gravity for combat. Punches won't be as effective as they are on Earth, since the puncher won't be able to brace themselves against the floor. Throws and flips won't be as effective, since the lack of gravity means they won't be smacked to the ground as hard - and even if they do, you don't get as much of a positioning advantage because you aren't able to immediately pin them to the ground, since you're still floating in zero-g.
The most efficient way to deal with a melee combatant in zero-g will be to get them into a choke hold and knock them unconscious that way. Course, they'll be trying to do the same to you. Due to how you need something (floor, walls, ceiling) to move off of/change direction, physical size will matter even more than it does on Earth - the larger person will be able to dictate the movement of the fight more than the smaller person.
@MozerShmozer has a good point in his answer (posted at the same time I'm writing this) regarding handholds for movement, but goes in the wrong direction with it, IMO. Since combat is basically devolving into grappling/wrestling, those handholds will be huge, but they won't make combat more familiar. They'll make it more exotic. The handholds will allow you to do all sorts of cartwheels, spins, and flips that would be impossible in normal gravity. Think of the hallway fight scene from Inception, only even more so (after all, there was still gravity there, it was just rotating).
Jump toward the person and grab them, you both fly towards the wall. It looks like you're going to pin them to the wall because of the angle you jumped at. They reach down and grab a handhold on the floor. Suddenly you're flipping. They let go of the handhold. You keep flying toward the wall, but you're spinning. Now you're going to be the one who gets pinned to the wall, so you grab a handhold to the side. Now you're flipping towards the side wall instead of the back wall. They grab....
And of course, the whole time you're both trying to choke each other. What fun!
An open hand combat martial style might emphasize Tui Shou or listening hands, or perhaps wing chun, all of which naturally would have to be modified for freefall, but nonetheless would emphasize the importance of disrupting your opponents' rootedness and redirecting your opponent's energy.
If I were to learn an animal style for freefall combat, I would choose Eagle Claw, which does not even require you to use strikes, but rather emphasizes pressure points and hand strength (picture an eagle's claw grabbing a pressure point in your neck or other part of your body and you can see why gravity is not needed).
Also, refer to Brazilian jiu-jitsu which emphasizes grappling and "ground work", which could be interesting mid air. You might even decide to rename this sort of thing "air work" which sounds kind of cool. The point of the jiu-jitsu style is to force your opponents joints to go in a direction they don't want to go. The freefall style would completely remove any sort of judo throws since those would be nearly impossible to pull off and wouldn't have the same effect in null G.
Overall, Okinawan and Japanese styles, which are linear, would be de-emphasized, and more circular, Chinese styles would be more important, as they are less reliant on linear momentum. And naturally any style that does not use direct force but instead uses grappling or pressure points would be an advantage.
I picture a two-handed setup, the right hand holding a relatively light sharp weapon while the left hand holding something much heavier, which would act as sort of a anchor due to its relative inertia. Most moves involve thrusting forward and attacking with the right hand while thrusting in the exact opposite direction with the left so that you remain relatively stationary and your weapon has the greatest possible force behind it. When defending, you could lift your left hand in front of yourself, causing your own body to move backwards. In your guard stance, both weapons would be held as close to your body as possible, offering the most options.
Usually you put your archers in the back row, and they shoot over your infantry's heads. Well, that won't work-- arrows won't travel on a curve in freefall. That puts your archers in the front row, where they are exposed. To minimize exposure, they can shoot downward between their legs (like they did in Ender's Game) to offer the smallest possible target, perhaps bent at the knee with arrow-proof shields attached to their shins, held in a manner where the arrow could be fired through the slim slot between them. God help you if an arrow from other other team gets through-- you would need a serious cup in your pants to avoid a very painful injury.
Since many people have discussed hand-to-hand combat, I will just skip to archery.
First off, there is no such thing as zero-G. Gravity is omnipresent in the universe. If mass exists somewhere in the universe, that mass' gravity is always present in the universe. Gravitational pull is lessened over distance, but never absent. Super low levels of gravity is called micro-gravity.
Lets take a look at archery in war on Earth. In your typical medieval/fantasy movie, you see the archers are safely shooting at the enemy from behind their allied ground troops. How can they shoot the enemy from behind their allies? Wouldn't they hit their allies? No, because they shoot over their allies. Earth's gravity pulls the arrows down. In order to shoot a long distance, the archers angle their bows above some number of degrees over being parallel to the ground in order to hit enemies on the ground. If you were to shoot an arrow level to the ground on Earth, gravity would pull it to the ground in half a second. The same goes for bullets. Rifleman angle their barrels upward to shoot farther, otherwise gravity would pull a bullet fired from a level gun to the ground in the same half second. To see this in action see this video from MythBusters, who did a great episode on this.
So now lets consider this in micro-gravity. Since gravity will not be affecting the trajectory (in a measurable way), every projectile will be travelling in a straight line. This means archers can no longer stand directly behind their allies. If they were, they would have to shoot level to the ground, and therefore they would shoot their allies in the back. Killing your allies is generally not an acceptable way to fight.
So what are the options? Well, in microgravity, the archers can be positioned above, below, or to the sides of their allies, as to not hit them in the back.
Of course, once the hand-to-hand combatants are locked into grappling battles, arrows become just as much as a hazard as they are an advantage. If two melee combatants were wrestling, how could an archer shoot an arrow and not risk hitting their ally? At that point they would be better off using melee weapons.
Another issue, albeit a less important one I think, is the distance of the arrow travel. Since micro-gravity will not pull the arrow into the ground, any arrow shot will travel a long, long distance. If the battle is inside a contained environment, obviously objects will stop them. If there is air in this environment, air resistance will eventually cause the arrow to slow over time and eventually come to a hovering stop. I don't know how far that would be, but I cant imagine air would slow it down in a significant amount of time. This means the archers need to be aware that any arrow that miss their intended targets will continue to be deadly projectiles for a long time.
Fighting in Zero-G is like fighting in an ice rink with soap bars tied to your feet intead of ice skates.
While floating in space, either on your way to your next goal or because you got thrown there, you are a sitting duck. You are absolutely incapable of changing direction or orientation once you lose contact with objects of close to your mass. If you are especially unlucky, you got thrown into the air with a spin, so you'll be far too busy trying to figure out which wall you are actually heading for to give any thought to the landing, let alone anyone taking potshots at you. Firing any heavy weaponry will just worsen your spin problem and reduce your fire power by a lot since you can't take the recoil.
Corollary 1: Outfit yourself with thrusters
They give you back at least some control over your air time. Correctly designed, they can bring you out of an uncontrolled spin (I imagine a lot of computer aid because timing such complicated maneuvers manually will be quite hard) and get you out of the air as soon as possible. Maybe they can even help you dodge in midair. Of course, the thrusters should not be too bulky and should have enough juice to reliably last the entire combat...
Corollary 2: Make Your Enemie Fly
Throw them in the air so that they can easily be picked off by your snipers. Or so that you can fry them with your Laser Pistol. Or just to get them away from you -- depending on how far the next wall is in the direction you're throwing them in, they won't be back any time soon. Even if they do have thrusters, they first have to stop their momentum, and then gather positive thrust again to fly back to you. That gives you lots and lots of time to prepare to receive them.
No matter what you do in Zero-G -- everything has a reaction. Throw a really fast punch? You will be thrown into a slow backward spin. If the punch actually connects? Half the force of the punch will go into accelerating you backwards. Swing a heavy sword? The heavier the sword is, the more you will be swinging yourself, too. Just imaging floating under water and swinging a sword there. Good luck with aiming and exerting enough force to actually get anything done.
Corollary 1: Get a grip
Use gecko shoes, magnetic clamps, some really high-friction rubber soles, handholds -- anything to get a good grip on the next best heavy object. It means you're not on your lonely own anymore, weighing less than 100kg, but you've got the entire mass of a heavy object behind you. The force equation is F=ma . You can't get rid of the reaction force (F), but you can certainly lessen the acceleration (a) you experience by increasing your mass (m) by adding the heavy object (best case: you plus all 300 tons of the ISS).
That way, you get rid of the nasty backwards acceleration and can make use of your leverage to either throw your enemies into the air (see Rule 1), or make use of whatever Aikido or other force-redirecting martial art you know. Because even they need at least some leverage to work properly. Your rubber soles might not give you enough friction to power effective punches, but force redirection should be possible. Or, you know, give you a good enough stance to deal with the recoil of knifing them somewhere.
See also Enders Game. You need to account for all three dimensions, and that you can station snipers on the ceiling just as well as on the ground. Especially in defensive situations, you need to make sure that the fire ranges cover all approaches, even from air.
Since recoil of any type is bad except for solidly mounted defense stations, projectile weaponry is pretty much out for Zero-G combat (and yes, plasma weapons emit gas plasma, so they are out, too). Heavy bludgeoning implements like clubs or large swords are out too, unless you can make sure you've got a really solid anchor to the ground and are unlikely to be pulled away from it.
Very sharp knives are better, since their damaging power mainly comes from the edge. Even better would be hand-held tazers.
Whips or hooked weaponry could be an option too if you do want to have the option of playing with your enemy like a yo-yo (throw them into the air, hook them back in as damaging a way as possible, throw them again, etc.). Whips especially could help with redirecting momentum when close to a surface.
It is also very important to have some kind of mid- to long-range recoilless weaponry to make use of your enemy's air time. Candidates for that are beam weapons of any kind like laser pistols, phasers, microwave weaponry, etc. Just make sure it is strong enough to fry human flesh while leaving your space station intact...
One-on-one combat should look a lot like Asian wire kung fu for movies (e.g. Hero) if there are no long-range weapons involved. Just fewer punches. Gather energy for powerful punches by thrusting yourself at your enemy from the opposite wall, keep the enemy from leaving your range of influence with whips or hooked weapons, or never let the enemy leave at all by immediately applying grappling techniques. With lasers, it will be more of a hide-and-seek with no close combat involved it all.
In zone defense, defenders have the advantage if they can make sure the attackers have to cross large stretches of air time. If the attackers can mount some kind of shield that lets them float across the empty space unharmed (if possible, with enough mass to make use of "Get a Grip") and they either manage to establish their own 'solid ground' close to the defenders or actually land besides the defenders, both are on even footing again.
General strategy for formation combat should probably take advantage of today's modern infantry strategies -- small highly mobile combat units entering from multiple angles, more like guerilla warfare than old-fashioned army clashes. Especially since those combat strategies will be applied inside a space station / ship / ... and not in empty space. So, think urban warfare instead of D-Day.
Given the parameters, I suspect that the key issue is NOT getting in an hand to hand combat situation. The second part of the question; dodging arrows is a bit strange, but we might assume that everyone has gone to Bigelow style fabric habs, so the use of firearms is a huge no-no inside any spacecraft or station.
Answering the second part first, any combat will involve staying near a solid anchor point, or moving in two man "fire teams" as a minimum. When you see the enemy raise a crossbow in your direction, you push off rapidly to foil his aiming solution. The two man team push off against each other, so instead of one target, you have two.
If you are unfortunate enough to lose your own crossbow, or end up being manoeuvred into a grappling situation, a rope becomes your friend. Using a noose or trying to wrap a coil around your opponent both restricts their movement, while at the same time providing you with leverage, and a "base" to take off from, if required (pushing against your immobilized enemy much like you would push off against a fire team partner). Obviously a simple rope can be enhanced, and you could see lassos, whips, bolos, ropes coated with a sticky substance (a "smart" sticky substance which does not stick to you would be better), grappling hooks, ropes made out of chain or high strength substances with cut and abrasion resistance...
I also suspect that other grappling aids will be developed, such as gloves with sticky pads on fingers and palms, shoes much like the the ones pictured here.
Uniforms should be very tight fitting and offer no obvious way for others to grab you, so no shoulder pads, molle loops, pockets with big buttons and so on. Think of a ninja suit adapted for space. This leaves the rather obvious question of how you carry anything, but if the space marine is carrying equipment in a separate "dry bag", they have both the resources close at hand and one extra object to cast off from to provide emergency mobility, something like this: https://www.youtube.com/watch?v=IbZl16rSRgg
There is actually an operations guide for the International Space Station for the situation if a crew-member became hostile, under the topic "acute psychosis or suicidal behavior". The recommended action is to grapple, and tie the hands and ankles up with duct tape. They are also equipped with tranquilizers for such a situation.
A somewhat more detailed explanation is given in this answer, among other things it also discusses subduing someone in micro-gravity in a cramped area.
First, a return to Obsidian blades. Most close quarters weaponry (swords, axes, daggers... maces, flails, clubs) rely on a forceful strike. Not gonna be easy in zero gravity, where a lot of the power of the strike would be spent pushing combatants away from each other. Unless both were well braced, the blow will not do much - for the same reasons turning into a blow makes it much less effective.
The other thing that can do damage is sharpness. A very sharp knife can slice even with very little force behind it, from the sharpness of the blade and simple lateral motion. It is the same technique that people use to cut delicate foods in a kitchen, gently drawing the knife across and letting the sharpness of the blade do the work. I can press a blade down onto my hand with some force and escape injury, if it doesn't slide sideways, or flick it across and draw blood. Much less effort to create an injury by flicking, than by trying brute force or grappling when force must act on both parties.
The major advantage of metal blades over stone is the resilience. Stone will shatter under force, maybe even if it is dropped, it is brittle. Metal is duller, and must be often sharpened to take an edge, but it can take more force, and be repaired if it gets dented. With much of the force taken out of the question (even dropping is unlikely to break without gravity's acceleration) I expect the greater sharpness - and obsidian is much sharper than steel - to weigh heavily on what makes an effective weapon. if two people are relatively close to each other, one can lightly draw an obsidian weapon over the other to cut into them, without the force of the blow forcing them apart.
So I would expect delicate precision to trump forceful blows in close combat - they're more efficient at injuring the enemy, instead of tumbling the opponents all over the place. A few ounces of force to swipe and land a cut, instead of shoving and tumbling away to give a bruise. Short to at most medium length blades, and as light and maneuverable as they can be made would be best, since you need the fine control a lot more than the momentum from a heavier blade and reach from a longer one.
Projectile weapons like arrows should still work, they injure by speed and sharpness, and the force that gives them speed is in the flexibility of the bow and the leverage between the arm and body of the archer. I expect aiming will be trickier, as people are floating around, but there's no reason the weapon won't work. Throwing knives might be trickier, they would need more force to give them speed, in order to be effective.
But really, one of the best projectile weapons would be blowdarts - they would be murderous in this kind of setting. Their speed comes from the breath and pressure building against the pipe walls, so they don't depend on their user finding good leverage, and if the darts are sharp they can penetrate quite deeply (like, a quarter inch into wood, which is much harder than people). Even plain darts can be dangerous if carefully aimed (deadly aimed at eyes or throat, still quite serious aimed at belly or various soft and squishy places on a person), and such darts were often poisoned for fighting or hunting with. Longer pipes mean more accuracy over distance on earth and less maneuverability up close, but without gravity to warp the trajectory I expect even a shorter blowpipe could be quite effective. Remember dodging can be quite trick if the target isn't near a wall.
Weapons with cords, like flails or whips or bolas, or even simple slings - I'm not sure. On the one hand, they can extend someone's reach and can probably be given extra force through body-generated momentum. On the other hand, without gravity I'm not sure if the momentum will build up or dial down. A sling, for example, can be given a lot of force by spinning it around before letting the stone fly - but without gravity to pull it down, or out, or away from the swinging hand, making the cord tight enough to move all parts of the weapon, it might just rotate without gaining speed. Same with a bola, swinging sideways. I'm imagining a forceful swing that travels down the line and twitches the weight on the end. So, might be excellent, might flop - I think a shorter cord might work better, since all the force has to come from the hand or arm and its more controlled, so it is much less of a force multiplier than if a longer cord will work. Maybe the ninja-style kunai which can have short cords attached, or short lines with grappling hooks.
Leverage. Well, hand- and foot-holds are well and good if one is near a wall, but combat probably shouldn't depend on being near one, when someone's life is on the line if they're not. I suspect there will be grappling lines of some sort, maybe swung (if the longer lines can be controlled), maybe shot out - a mechanism like a crossbow might work if a grapple gun is not allowed, or maybe simply thrown like darts. Depending on the material of the walls, this be something sharp to dig in and hold, or something very adhesive to stick fast.
The line can then be an anchor-point to the nearest wall, used to dodge or navigate by adding sideways momentum, or directly used in combat by being landed on a person or object to capture them and drag them back, tangle them in the lines, slice into them if sharp or get adhesive over things if sticky (like landing on the junction of a weapon sheath so the weapon sticks to it). Given their use, I would expect people to have several such lines available for normal navigation as well as combat, and because retrieving them might not always be feasible, someone cuts the line, someone needs to maneuver and drag their opponent in, etc.
I'm not sure about formations and such, I expect everyone would need a lot of room until they know how this kind of combat works. There will be a lot more movement as fighters drift around, bounce off walls and each other, keep moving till something stops them. Maybe they could work in pairs or trios at most, so they can use each other for leverage and watch each other's backs... but more than that would have a really high chance of fouling each other up more than helping.
Someone might be able to brace more people in a smaller place, or tether people at intervals to a wall if they're on the defensive, know or can modify the space. A knowledge of local leverage could make them mobile within their little areas, letting them work together more densely, and if they've got projectiles, they can produce something like mass fire, and it could be quite effective at defending a space... but they also can't dodge too much to avoid any return fire, and if the enemy gets too close to their position, that bracing in will make it harder for them to get out and avoid being trapped.
I imagine there would be multiple styles incorporating combined arms tactics just like we have in earth surface gravity. You would want ranged weapons to shape the battle space, bladed weapons for their lethality, and impact weapons for their momentum.
A fighter that relies on speed and agility would forgo heavy armor, but this would make them vulnerable to ranged weapons (darts, arrows, etc.). I would expect sword/knife wielders to take this tactic. They might carry a small shield like a buckler as defense against ranged weapons, but this would be a huge compromise and provide limited protection. However, they would likely be the major damage dealers.
A fighter that relies on bulk and mass would armor up to protect against projectiles and blades, and would likely prefer impact weapons (clubs, hammers, polearms). These would be the front-line warriors and the ones who provide cover for their allies. They might carry large shields (kite, etc.) or stick with 2-handed weapons with a long moment arm.
And, of course, the support fighter would use ranged weapons to deter the lightly armored warriors from closing too quickly, massing, or approaching lightly defended targets. They would be able to project force across the widest distances the most quickly, so you would always want them available, but away from the front lines.
It is quite likely that all fighters will have some heavy tool/weapon on their person at all times. Why? Because if you get stuck away from a wall or other leverage point, and someone fires a projectile at you, you are a sitting duck. If you carry something with heft and some distance, you can use that to at least dodge a short distance. In an emergency, you can throw the object for even more momentum (but obviously, that puts you at risk until you can recover your counterweight).
For this reason, longer weapons may be preferred to shorter ones, depending on what kind of attack is favored. A halberd could be a fearsome weapon in the hands of an armored defender. A fast striker who closes with a sword would have to contend with the pointy end and the prospect of their own momentum impaling them. A heavy blade on a long pole would give the defender a nice moment arm for pivoting to avoid projectiles even in open space. Or, if the defender or his ally is anchored, the halberd could be used to pass momentum/anchoring to/from the ally.
Alternatively, dual wielding shorter weapons would offer even faster reaction times (perhaps necessary for sword wielders with lesser armor). A sword attacker would want a defensive sword to parry spear/halberd thrusts, or a buckler to fend off ranged attacks. Against lightly armored foes, a sword could be quite deadly, but against the armored defenders, a vulnerable strike would probably be required. Even so, sword warriors may carry something like a flail to provide emergency maneuvering away from anchors. Or, they may carry several, attached to a belt, so that they can make mid-course corrections by throwing weights at walls as they fly through the hallways.
One of the benefits of microgravity is that warriors can fly much like you see in stylized martial arts films (e.g., Crouching Tiger, Hidden Dragon), but the downside is that once "aloft", they cannot easily change course. Thus, having throwable attached weights which can grapple or just bounce off walls may give critical maneuverability for high-speed lunges at the enemy.
Finally, the ranged fighters would use their projectile weapons to prevent/deter flanking attacks and direct the opposing forces towards their strongest defenders. They would also make opportunity attacks against fighters that ended up in a vulnerable position while focusing on someone else. At some point, they may be forced to engage in melee combat as well, or perhaps they would instead go for high agility and lend their momentum/anchoring to their allies, relying on others to deal damage.
What I do not envision is much grappling. Bare-handed fighting is not likely to result in much damage. Any close-quarters combat will at least call for a knife, sword, or thrusting point. All momentum will need to be generated by arms and legs, with no contribution from gravity. Thus, slicing and piercing would be the most effective forms of damage, augmented by poisons, naturally. Blunt force could only be delivered by lucky blows or an inattentive defender. Any successful pure momentum attack would be twice as effective with a blade or point involved.
