Some heavily armed marines are dumped out of my spaceships to attack the other sides boarding parties. It is hard sci-fi so there is light armor to defend against shrapnel on the ships. Th marines have power armor and twin 50 caliber assault rifles bolted to their arms. They also have small missiles and welding torches to breach the enemy ships. Theses weapons are powerful enough to get through a couple layers of armor and breach a hull. In a confusing space battle with lots of ships zooming around in 3 dimensions and marines shooting lots and lots of bullets at each other and other ships firing PDCs like there is no tomorrow sensors will be overwhelmed. Is there any way to prevent lots of holes in both friendly and enemy ships?
Sensory Overload is a Bottleneck that can Happen in 3 Places
- your sensor becomes too overwhelmed with noise to make since of
- your user interface becomes too cluttered to make since of
- your focus becomes too wide to make since of
The good news is that computer scientists already know how to solve for these issues; so, with a little bit of clever design, you can solve your sensory overload problems no matter how cluttered your battle field gets.
Solve the Sensor Issue with Heuristics
If you have thousands or millions of contacts on radar, your onboard computer can not afford to track each one individually; so, instead you would have your system fall back on a heuristics algorithm. Basically instead of trying to isolate thousands of bad guys from the giant pattern of static they create, you instead use the static pattern itself to approximate about how many contacts it makes to create that pattern. Friendly units will all use synchronized transponders so that you can get a familiar pattern out of battlefield background noise; so, if your computer sees a static cloud that would take about 5000 troops to make, and the intensity of the friendly synchronized transponder noise represents about 2100 signals, then with very little processing power you can deduce that you are looking at about 2100 friendlies engaging 2900 enemies.
Solve the Interface Issue with Heat Mapping
In situations where you only have a few targets to track it makes a lot of since to place individual markers for each enemy, but as the battlefield becomes target rich, this kind of display can become very cluttered to the point everything just becomes a massive blob. Heat Maps are a common user interface designed to turn complex blobs of data into simpler displays by summarizing groups of data.
So in the example below, the 1st output tells you exactly where everyone is, the second one is kind of useless because without knowing how spread out all those overlapping dots are, you can't really estimate how large of forces are and where, and the 3rd shows how heat mapping can turn a lot of data points into a summery that would allow this person to tell generally where he is needed at a glance.
So, when your sensors become overwhelmed, your input falls back on heuristics and estimates, and your display switches to heat maps.
Solve the Focus Issue with LoD Balancing
Have you ever noticed how Google stores a map of the whole world, but can always send you the map you need over a simple internet connection pretty quickly? This is because it uses scalable LoD (Levels of Detail). The more you zoom in on your map, the more detail it gives you by excluding information that is outside of the scope you are actually looking at.
This is how you will specifically solve your friendly fire issue. Instead of scanning the entire battlefield to see if your shots will hit a friendly unit, you just scan a small conical section of the battlefield representing who might end up in the way of your shot. So, let's say are in the middle of a 3-d battlefield with 10 million combatants. If you only scan a 3 degree cone in front of your gun, then statistically there will only be about 17 combatants to worry about; so, if you use an AI safety mechanism similar to that described in Henry Taylor's answer, you will reduce the overwhelming complexity of the battlefield to a much simpler problem that you can compute.
To make sure your system is truly never overwhelmed, you incorporate an optimal stopping solution into your computations. So, you might start off calculating for everyone in a 1 degree arc, then 2 degrees, then 3, etc. until you hit the maximum computation time allowed per cycle, then you stop. This way, you might have a slightly reduced firing discrimination in a particularly crowded situation, but the system will never totally fail on you from being overwhelmed; so, at least most of the time, you are only going to hit bad guys.
Lastly, don't use 50cal cannons as you main weapon
In zero-G, armor weighs nothing meaning that heavy armor is more practical. While it still takes energy to accelerate in space, there are way fewer forces working to slow you down that you need to overcome.
This related question shows how easy it would be to stop a 50 cal round using armor that can fit onto power armor. If you assume 50cal machine guns are common for power armor, then you should ask if power armor can also stop them. As it turns out, it only takes about 30mm of relatively lightweight fiberglass/ceramic composite plating to stop a 14.5mm machine gun firing armor penetrating rounds, and a 50-cal only fires 12.7mm rounds. So stopping it with current day materials on power armor is very doable, and if you look at current trends in how armor and ballistics are progressing, it will only become more doable if you assume future tech. So, if you go with traditional ballistics, you will probably need to consider using something more like a 20-25mm auto-cannon if you want to make sure you can punch through more armor than your enemy can practically wair.
But... more importantly, you need to make sure whatever primary weapon you are using is not punching holes through your ship as an auto-cannon likely would. For this you should consider a primary weapons that can kill through armor without penetrating it. For this I suggest an HEL (High Energy Laser). In this related question I explain in detail why High Energy Lasers are great for killing people in power armor without risking putting holes in your ship, but in short, it is because you can cook a person inside of armor LONG before you make things hot enough to melt through a spaceship's hull. Also, these work well for your hard sci-fi criterion because they are already a thing and they work much better for your zero-G environment than balisitics because of recoil.
You may still want some high-explosives for breaching doors, but the use of HELs as primary weapons will mean way less accidental damage to your environment which in turn means way less collateral damage to friendlies.
Physics: Use lighter or slower rounds that cannot penetrate ship armor.
Number of rounds: Limit the amount of ammo per engagement. After all, it needs to last the entire campaign. Example: Use a 'burst' mode instead of full automatic fire.
Have rounds self-destruct after a specific range or time.
Just equip your assault rifles with artificial intelligence smart enough to calculate the trajectory of the projectiles (including ricochets and deflections) in realtime and then put the trigger under that AI's control. If the soldier attempts to fire a bullet in a direction which would hurt any of the precious ships, the gun doesn't fire.
Your ships will be fine.
You specify 50cal for your space marines. That is projectile size. Let us briefly consider space vs planet surface as they affect projectiles and armor.
1: No gravity = no drop. As compared to planetside where projectiles fall fast under gravity, your projectiles travel in a straight line.
2: No air resistance in space. Your projectiles will not have to travel as fast because they will not shed energy to the air.
Your 50cal projectiles as used in space are much slower than the same projectile used planetside. I imagine them as comparable to a Minie ball as used in the US Civil War. These are antipersonnel weapons and they dont have to be that fast because they will have the same muzzle velocity when they hit as at the muzzle and range is effectively infinite. They will still pack a huge wallop if they hit an enemy soldier. They are not for use against vehicles; you specify the marines have other tools for that.
Armor in space. The best way to armor against a projectile is to gradually deplete its kinetic energy. This takes time and time requires space. Your ships have hard (but not brittle) outer shells over a few dozen meters of helium-expanded styrofoam. This slows and stops any projectiles except those designed to get through. In space it does not matter how bulky your ship is, only how massive it is.
But if you are a human, bulky still matters in space. The space marines have armor but they cannot tote around the bulk of all that styrofoam. Their armor is hard plates of body armor which fragments on being hit, and so is depleted with use.
I will take your statement that "sensors will be overwhelmed" as fiat (though it's definitely debatable). In the abstract, that means there will be no closed-loop control on targeting. I'll leave open the possibility for sensors in smart projectiles. That leaves a couple of avenues to explore:
Plan Where the Friendlies Will Be
A creeping or rolling barrage, common before the widespread of radio communication and fire control, utilized a set plan on a schedule to say where the enemy will be (where you can shoot) and where the friendlies will be (where you can't). Flares and other visual signaling methods can also be used to "shift" the fire when predetermined break points are reached in the plan; with advanced technology there are many more options for doing so ("we'll send the signal when we reach the surface of the enemy ship, do not shoot at this area afterwards" or "we'll be suppressing enemy defenses in this area, do not move into it until their power signatures are destroyed").
This technique is shown during the Battle of Ragnar Anchorage in the BG miniseries, where the small spaceships (Vipers?) are told at some point to stay out of the big spaceship's fire corridor (there's a line of dialogue to that effect, maybe, but I can't find it).
If your projectiles contain some kind of shaped charge (not a bad idea for weight savings), they could "stick" to whatever they impact for a moment and send out a close range friend-or-foe interrogative. If that answer is "I am your friend" (encrypted, of course) the projectile renders itself safe, exploding otherwise.
See my answer here for a more in depth explanation of how such a smart projectile would work.
Just use IFF
Identification 'Friend or Foe'.
Your in space in a zero G setting. Regardless of anything else space suited soldiers (assuming anyone was silly enough to send them outside) are just as hard to hide (stealth) as space ships are. The suits are going to radiate heat, just sitting still, even more moving and firing (with the possible exception of cold gas thrusts). Plus there will be em and laser emissions that can be detected. So if you can't hide easily you may as well just declare your presence - at least to your friends by 'pinging' on their HUDs with an encrypted IFF signal as often as the tactical situation requires. Could be on a timed basis or just whenever a friendly targets you with their weapons systems. You simply set the parameters (and codes) before commencing the op.