I've been looking at the idea of Multiple Telescope Arrays, but there's an aspect I can't quite figure out. I have the angular resolution part down, and it's absolutely nuts how far away you can see even small things given a wide enough aperture. What I'm not sure about is the amount of data involved and the sort of processing power required to handle this. High def images can get pretty hefty in terms of bytes, and a telescope array capable of mapping coastlines across interstellar distances is going to be quite hefty. Calculating the bytes in an image is as simple as multiplying their pixel count by 3, but how do you calculate the amount of pixels an array would capture? How big of a computer would you need to process the data from such a massive telescope array?
I'm mainly wondering if a Multiple Telescope Array measured in astronomical units needs an orbital megastructure-sized computer/AI to handle it, and how that would scale up even larger. Timeline of development for such massive computers/AIs is an important aspect of the setting I'm working on, given their relationship to other high technology. So is the sort of information such a system could tell us about the galaxy, and when we learn that information (such a system could easily spot alien civs, after all).
Finding out how powerful telescopes can get in such arrays strikes me as a super underutilized concept and I want to try and get it right.
Your instincts are right this is computationally complex, however this would be manageable with current computing infrastructure - it may take a few months to develop a photo to it's maximum potential from the "take photo" command but this work can be pipelined: the array is capturing one "image", transfering the previous one, and the ground computers are rendering the one before that.
You could definitely put a few hundred spacecraft across 1AU or larger distances to capture distant objects at high resolutions.
So - they wont be using RGB8
You won't be using 3 bytes per pixel; two main reasons:
all those wonderful images NASA publishes are falsely coloured. They will capture data from a sensor of some frequency in the electromagnetic spectrum, and map that to a colour. It may be xrays, it may be radio waves, it may be infrared, ultra violet, or it may actually be visible light. It's essentially a 2D array of intensity that is coloured and turned into an image later.
256 intensity values isn't enough resolution when dealing with subtle variations in very bright or very dark data. I'd be thinking 16 bit minimum. If your use case is imaging continents in neighbouring star systems I'd be trying to count raw photons if you have a sensor that makes that possible. 64 bits may not be enough in that case.
You may even want to send more data than just intensity of that frequency per pixel. When capturing from sensors at super high resolutions like this every pixel may not be captured simultaneously. If you have spacecraft travelling at Kms per second looking at planets travelling at kms per second a few ms difference in pixel sample time matters. This may be calculatable or consistent between captures but any subtle variation in it (processor scheduling? Busy devices? Many such things would through the timing off). You may want a time offset per pixel.
However you dont need to transfer all this data between photos. Typically spacecraft receive instructions for dozens of pre planned data captures, they save the data to an on board hard drive, then when completed they send thumbnails of everything they took, and then slowly over the following months as power and transmit windows allow they compress and then send the bulk of the data.
You can use progressive compression technique (think jpeg images that start blurry and as data arrives they increase in resolution). That exact technique (inverse discrete cosine transform) will work on this data quite well - if the data is truly interesting you can get every last bit losslessly through residual differential coding, however if the capture wasnt what you wanted you can learn that sooner without wasting bandwidth.
How big is the image from each spacecraft?
What's the resolution of the sensor on each spacecraft? Your sensor will only be able to detect where the photons hit to a certain accuracy - and that accuracy determines the size of your image. By the time humanity gets to this level I have no idea how accurate a camera will be - this is just educated guessing.
Individual spacecraft will likely have multiple "cameras" with different technical specs (esp different filter frequencies) so this value may be fluid depending on what type of wave your capturing. Your spacecraft may even capture multiple images concurrently from multiple sensors facing similar directions to feed into the ground algorithm as multiple "virtual spacecraft" - a spacecraft with 3 cameras 100m away from each other on long sticks is probably cheaper and easier to launch than 3 individual spacecraft - and your final image will have 3 times the source data.
Any figure I give you would be a guess - I'd suggest 2^16 wide and high would be an upper bound on the useful resolution a futuristic image sensor could capture. (65k x 65k. 4.2 gigapixels per image).
At 64 bits per pixel per channel your looking at an upper bound of 32gb per frequency per spacecraft per photo uncompressed.
This could compress to a 1000 byte thumbnail or any size in between as the ground operator gradually gets confidence in what's been captured and request higher and higher resolutions.
The ground processing.
The filters for this are complex. Were talking convolutions and fourier transforms and other stuff I don't thinks worth going into detail over unless you plan to code this up.
But your talking at best cubic algorithms here - but as people have pointed out you dont need terabytes of ram to do this - you render it tile by tile from your source data. Youd pick a tile size that was convenient (try based on cpu cache size - maybe a 1024x1024) and then you read all the data off disk to build that tile (you need to read each input pixel multiple times but you can sort and cache and get this fairly efficient). You may be reading 50 terabytes of data per tile but from a fast current gen SSD that's only a few hours per tile - faster if you use multiple disks instead of one big one.
This can be done on multiple computers in parallel so the rendering speed comes down to how much server rack time can you devote to the task?
You could also run a low quality algorithm to work out areas of interest, and then only run the algorithm at full fidelity over the areas of interest. This could reduce turnaround time for a photo to days or possibly even hours.
Station keeping or position measurements is going to be very hard.
Station keeping these spacecraft is going to be very complex - each one is going to tend to go into their own orbit, and you need precise location information in order to run the image stitching algorithms. You can calibrate by taking photos of known objects but if those spacecraft are in motion relative to each other the calibration is immediately invalid. If you can't keep them still you need to know their exact relative position with sub mm accuracy - that's difficult across 1AU distances but not impossible - put multiple laser distance measurers on each spacecraft and measure multiple distances to neighbouring craft in real time - and/or a supercomputer attempting to calculate the N-body orbits (or we may solve the N-body problem), and this is plausible.