how would a spaceship helm "user interface" (flying controls) be designed in detail

Question

When flying a big spacecraft, one needs to keep several things in mind when designing the "user interface" used to control the spacecraft's flight path, or ultimately, the output of the thrusters/engines/warp field generators/whatever it is using.

How would the interface of a "medium-size" (think of what the main protagonist group of "the expanse" or "firefly" uses) spaceship look like to make it controllable by humans: Like a bunch of general-purpose Computers with software where one can click his path through a 3d map? Like a collection of thousands of switches and sliders, together with countless dials and numeric displays? Like a fighter jet mainly with a joystick and thrust lever? And especially for any software or display on a "normal" screen, how is the software user interface designed?

I imagine that when the spacecraft reaches a size where it's not controllable by a single person anymore, there needs to be some distinction between high-level flight path; and low-level control of the engines. Most of the low-level control is reasonably abstractable by computer control and autopilot. But what if the spacecraft needs to be controlled in a situation without long planning phase and big error margins? Eg. during combat or maneuvering an asteroid field, in contrast to "travel from planet X to planet Y on fastest path".

It will probably be something inbetween a nuclear control room (Fig. 1) and submarine helm (Fig. 2), but much more complicated due to more complex layout of a spacecraft, much redundancy and lack of reference directions ("ground"), no inherent "auto-braking" (no surrounding atmosphere or water) and unrestricted movement of the whole vessel along 6 degrees of freedom.

How would the "human interface" be designed to accomodate complicated maneuvers, calculate flight paths in space, and reaching a target (or evading enemy fire) a robust way and with acceptable operation complexity.

I'm looking to design some spacecraft-control simulation (not write a novel), but existing games always feel to me like realism has been sacrificed in favour of gameplay and fun factor. My goal is to have something as realistic as possible, and I'm looking for pointers and caveats when designing the helm.

Answer 1

Tightly Cramped and Optimized For Minimal Crew

It is ironic that in space the one thing that is extremely cost prohibitive is space. This is because life support is very power-hungry, high maintenance and delicate, so you want to have to use as little of it as possible. As well, food and water is heavy, cannot be reduced to take up less space past a certain point, and is one of the biggest determining factors aside from fuel as to how long and how far a mission can go. To counter this spacecraft need to be optimized to fit as much as they can into as small a space as possible and to utilize as few crew members as possible as well. A control center for a space vessel is probably going to be packed with machinery and have only as much space is required to fit a few crew members in a seated position.

It will probably be deep down in the center of the vessel where there is as much armor as possible as well. There wont be any windows, since in space combat will be occurring at thousands or even millions of kilometer distances and there really isn't anything looking out a window could help out with. You would probably have a seat where a pressure suited crewman is manning communications, navigation, and steering (probably heavily assisted with automation.) Steering the vessel will not be a dramatic joystick hauling "pull up pull up!" affair because maneuvers in space can take hours, days, weeks, or even months. The navigator will simply execute a series of computer commands which the computer will then carry out (like increasing acceleration 2% for the next 8 hours to optimize the effect of an upcoming gravity well slingshot.)

The next would be the engineer's station where he is seated to monitor the ships mechanical functions, fuel level, engine status, damage reports etc etc. The third would be the gunnery station where somebody operates the ships weapons, also probably heavily assisted by automation. Keep in mind the engineer is not a "scotty" who can whip up battlefield repairs under fire. Any actual repairs the ship will need are going to have to involve a spacewalk and wont be able to be done in combat or while the vessel is accelerating or maneuvering. All personnel on the vessel will probably be at least capable of assisting the engineer.

The next is the captain. The idea of a captain sitting in a big centrally located chair behind and above everybody gesticulating dramatically and trading verbal tirades and witty barbs with bad guys on a giant screen video chat is not going to happen. The need for a "captain" is going to be a bit different that tradition naval vessels. He will probably be a subject matter expert who can assist with any of these three guy's duties as needed or take over if somebody dies or is injured In combat. This is unlikely since anything that hits them will probably kill all of them or damage the ship badly enough to render such measures meaningless. Still, its good to have contingencies available since a crewmen can just as easily get killed or maimed during the ships downtime on accident. The captain will probably be seated into the now very cramped Control center to ensure that all personnel are operating optimally, convey mission orders, and generally assist with the operations from his command console as well as assisting with tasks when things get busy. It will probably look an awful lot like a space shuttle cockpit minus the windows. Everybody will be packed in like sardines and only have as much space as is required to operate their station and get in and out of the control center.

A war vessel will probably only require 4 people to operate in combat, and will probably have 3 to 4 times that number tops. These are Replacements for when battles or operations end up taking weeks or months and members need sleep but the ship still needs to be constantly crewed in case a surprise pops up. This is a watch rotation which ensures some redundancy in crewing and also ensures nobody is manning an armed war vessel after being awake for 72 hours.

Things change dependent upon the ship's function however, I am former military so I can tell you a general idea how combat operations work, but dont know a hell of a lot about stuff like transport missions or scientific expeditions. Civilian ships will probably have a bit (A weeeeeee bit, not much) more room, and carry more crew if they are performing something that requires more manpower and long duration's of time sine scientific expeditions could take years and even basic cargo runs could take months. But the life support sections will always be fairly small and make up a small portion of the ship in total. Realistic spaceships are mostly fuel and engines with a tiny payload and habitation section relative to the size of the amount of fuel required to accelerate and decelerate it all.

The size of a ship is not related to the size of it's control center. A massive ship can still be controlled from a tiny heavily armored room nestled deep inside the armor and superstructure of the vessel. A cockpit like this could control a ship the size of a cruise ship.

Answer 2

User Interfaces are hard; really hard.

Humans and ships, of any kind, at any scale, form two part systems. Together, they achieve whatever missions/goals the ship is meant to fulfill. Humans are exceptionally good at planning, dealing with operational exceptions (where things just don't go as planned). They stink at dealing with boring, mundane, repetitive functions. They lose interest quickly. Happily, machines are exceptionally good at dealing with the boring, mundane, repetitive tasks that humans stink at. Machines are generally poor at dealing with new scenarios or really exceptional circumstances.

Humans also have limited cognitive load capacity. There's only so much information you can stuff in their heads at one time and their attention is strictly singular; they cannot manage more than one task at a time. We also remember that larger and more powerful systems are almost always more complicated than smaller systems. These more complicated systems easily exceed the cognitive abilities of a single human.

The UI will also depend heavily on the design and function of the system being controlled. For a fusion power-plant that runs a turbine for power, the RPMs for that turbine would be critical information. For a fusion plant that just generates electrical power without a turbine, there are no RPMs to monitor.

Design Process

Designing User Interfaces is hard. The search space for user interfaces is huge. Given the complexities of spaceship operations, it's nearly impossible to reason about what will be needed or not needed, a priori. Part of this is just trying different user interfaces to see what works.

Designing user interfaces for a spaceship isn't far removed from designing the ship itself. The two processes are highly integrated.

UI Design Process

1. Determine what systems are necessary for this ship to function. Assuming a spaceship, this will be systems like weapons, sensors, power-plants, life support, navigation/manuevering and so forth.
2. Design these systems; what components are required with rough approximations for tolerances on these components.
3. Figure out the first order approximations for how these various systems will interact with each other.
4. Determine what can be automated. The more that can be automated, the easier it will be for crew to operate and the lower the crew requirements. This will include figuring out what information needs to be filtered/translated so that humans can reason about it. For example, feeding a human raw spectra data on the exhaust plume of an incoming spaceship isn't likely to be as helpful as indicating "This is a hydrogen-uranium torchship with a 1GW torch running at 101% expected capacity".
5. For all the tasks that humans should handle, design a user interface to present the appropriate information and the right controls to manipulate that system. It's beyond the scope of this answer to describe what is right and what's wrong in UI design. Large companies such as Microsoft, Google and Apple have spent decades and untold sums developing and publishing user interfaces and user interface guidelines. This is an exceptionally hard problem.
6. Gather info about what works and what doesn't work. What info is needed but not present and what info is unneeded? Iterate over this step as often as needed.

Dealing with Failure

The user interface will also have to deal with the failure of various components. What do you do if your main tactical UI fails? What does the power-plant engineer do when he loses main power but needs to monitor all the auxiliary power-systems still? For critical systems, there must be a way to reason about the system state and control it when the spaceship has exited normal operating conditions (such as heavy damage during a firefight).

You must also contend with UI failures where the system/UI does not prevent dangerous operation conditions. The Therac-25 is an excellent example where a UI doesn't present the correct information and the system itself doesn't have necessary safe-guards. As the ship/UI designer, you have to prevent these kinds of events.

Cognitive Continuum

As the UI designer, your job is to present the information required to the crew so that they can compete whatever their mission is. This will depend completely on the type of work that each crew member is responsible for. Navigators will require different information than powerplant engineers. They will also perform different functions given their responsibilities.

Humans can only think so fast and can only process so much information and only have so much tolerance for repetition. Therefore, the machines/computers in the ship need to handle the boring stuff and also to filter the absurd data quantities so that the human operators aren't overloaded.

Design Considerations

Someone has written a game around hyper-realistic spaceship controls. Scott Manley did a walk-through. There's also startup procedures for real aircraft that are available too by DevilDogGamer. Granted, both of these demonstrations are for single pilot craft but it demonstrates the complexity that is easily found in real craft.

As we don't have fictional spaceships yet, any user interface for those ships will be purely speculative.

Answer 3

The exact controls would greatly depend on exactly what kind of ship they are for

I would think that in most scenarios, a supercomputer would be responsible for most of the detailed operations, with a human pilot giving basic instructions. For instance, a pilot might enter a command to go left, while the computer actually calculates the exact specifications of the maneuver and controls the engines, scanners, and everything else.

Still, there would probably be buttons and switches for everything, in case of computer failure. The pilot would still need to be highly trained, just in case something goes wrong.

And of course there would be a self destruct button, because you just can't have a spaceship without one.

Answer 4

Think iOs apps for Star Gazing

I've been thinking about this a lot - in the future if you're in a ship there is no need to:

• go 'left' up or down, or any cardinal direction, because the AI computer would simply plot the most optimal course for you, and there is no way a human could remotely calculate themselves by hand (or in their head) the complexities of orbital or galactic mechanics
• operate any ship functions, because the AI computer will automatically detect any defects, proceed with any redundant systems, log any issues, adjust any environmental controls beyond simple lighting and temperature

So that just leaves you to control the 'purpose' of your voyage - ie. your destination. And in this regard we already have good hands-on experience today, with smartphone / iPad interfaces for skywatchers.

I would imagine a search function, with perhaps quick data available on your destination, maybe even in a hand-held device interface like:

Or you can even go more simple (for kids to navigate):

All other information is simply not relevant to the occupants, or too complex for the occupants to make any informed decisions.

Answer 5

Humans can deal with much more complexity than the solid surface of a control board will permit. Add to that intelligent UX design such that things you need to see are only visible when you need to see them and we have a winner.

But, what does this look like? For one thing, you need to detach from the idea of control boards completely. Too limiting.

Our sister site, SciFi.SE has the right idea. Use a Matrix-esque control system where the physicality of the UX system is non-existent.

(Click for larger view)

Once you have a 100% software-controlled UX you can design it to take advantage of what humans can do, regardless of what the ship's needs actually are. The underlying computer does all the heavy lifting.

So, in an ideal world, the physical UX is a comfy couch with a head-band with a lot of wires attached — or with a probe that plugs into the back of your neck. No windows, no switches, no leavers, not even a readout. Everything is direct-to-brain.

Answer 6

The other option is AI controls the ship and people just say what they want.

AI would have faster reflexes and better control than any human.

It's like asking what controls are needed for a self drive vehicle. Beyond telling the vehicle where you want to go, no controls are needed.

Answer 7

I think Pixar provided video documentation of this in “Lifted”:

It’s hard to think of a more flexible user interface than thousands of paddle switches...

Answer 8

I'd like to hit a tangent. That's a prime example of date-stamping by electronics. We are talking about, say, a bridge or a spaceship. This seems in-line as a classic Sci-Fi item, and SCi-Fi is old. But the how depends a lot on the time of writing / shooting / making up the design.

• Like, in the 50s that would be a lot of clutches and wheels and tiny green screens. And the navigator gets to print out the exact route from a huge computer. The input to the latter was fed in punch cards.
• Or in 1900s that would be wheels and compasses and telephones to call the engine crew.
• In 70s that would be keyboards and CRT monitors everywhere.
• In 80s that would show the cyberspace the way they imagined it then.
• Somewhen around 90s that'd be something like a jet fighter controls.
• Nowadays that would be some kind of a virtual reality head-up display.
• A typical Sci-Fi vision (from 90s up) is some kind of a neurointerface, for example, one that mentally replaces a limb. "When I pilot a ship, I forget, I have a left hand." (c) Lafier from "The Crest of Stars".

---

So, the proper answer on your question depends on the aesthetics you are aiming for.

Answer 9

Here's a presentation video by Rolls Royce where they showcase their plans for a shore based control centre where operators monitor fleets of AI driven ships. They want to emphasise its "high tech but doable with current technology" nature hence drones, touch interfaces

Another presentation this time a holographic display that is actually a projection on a curtain of finely misted water

A different kind of interface a DJ mixer that uses perspex blocks to mix and manipulate sounds rather than knobs, switches and sliders

Answer 10

It's extremely difficult to manage how to fly a ship depending on WHAT LEVEL of dimension you're talking about.

Apollo was the closet we ever got to real term long distance control of human spacecraft endeavors.

Question is based on What level of technology and dimension you control....In space travel there are six dimensions

• Pitch, yaw and roll, The classic dimensions in typical aeronautical control....
• Asscent/descent (Straight up/down)
• Side by side shifting
• the last two in a differing plane of orientation It's safe what's Comfortable; Joysticks, wheels, yokes, palm balls, computer mouses, even touchscreens.

Least say there'd be an quantity of screens at one's disposal and lots of people staring at them.

Big windows like many scifi series only make sense if you're landing/docking or conducting affairs in close proximity to other ships. Vast quantity REAL spaceships are controlled by Inertia and steered accordingly, extensive control and automation. An independent spacecraft in Deep space would need host of technologies and controls in order to function. Space is Far too large for pilot to "Eyeball" navigation. So your ships bridge and cockpit is full of screens.

1. NAVIGATION: A network of possibly three-dimensional maps in reference to position of itself and objects in fictional plane you assign. And scale mapping depending on distance (Lightyears, AU, Miles, feet)