In wartime, megascale data centers may make way for distributed architectures

Link: “Data embassies” and safeguarding digital assets during wartime, by Rina Chandran in Rest of World

Among the targets in the war between Iran and the US have been data centers. AWS was hit by drones, and Iran has threatened to target US tech. This piece makes the point that these buildings don’t just store vast amounts of civilian customer data: increasingly, they store military data, too. That make them an even more attractive target and makes the security consequences of an attack that much worse.

Meanwhile, data centers — including here in Pennsylvania, where I live, as well as Chile, India, and many other places around the world — have been the cause of significant objections from local populations. They push energy costs up, have a serious environmental footprint, and can even change the local climate.

So why have these giant megascale data centers at all?

““It’s very possible that we see a move away from hyperscalers to small data centers for greater safety,” [Viktor Mayer-Schoenberger, professor of internet governance and regulation at the University of Oxford] said. “Lots of small data centers with randomly distributed backup copies of data are more resilient – but harder and more complex to build, more costly to maintain, and less effective, as data needs to be kept up to date not just in one or two centers, but in many.””

The latter half of Mayer-Schoenberger’s claim is true if we cling to the same architectures. But if we embrace more decentralization on the architectural level as a founding premise, some of these inefficiencies become less of a problem. It could even be worth it to companies like AWS to build new underlying services that make decentralization easier: abstractions that allow data to be sharded across distributed data stores, and that make secure communication between distributed nodes easier.

That’s clearly necessary if we move to smaller data centers: a smaller venue can’t simply hold a copy of all the same data as the larger ones but in more places. It also opens up the possibility for mesh application layers rather than the monolithic mainframe-style architectures we’ve mostly seen on the cloud. Behind the scenes, cloud services are a sea of proprietary micro services and components; building more distributed architectures could more easily allow each component to be built, hosted, and supported by different entities.

Regardless, the change is an interesting thing to think about, and the cause for it is sobering. What does data infrastructure look like in an increasingly antagonistic world — one with more war, accelerated climate change, and authoritarian threats? Those considerations will need to be built into the internet at a backbone level, and into its applications from the ground up.