Werd I/O

The Ones Who Walk Away From Omelas, by Ursula K. LeGuin:

They all know it is there, all the people of Omelas. Some of them have come to see it, others are content merely to know it is there. They all know that it has to be there. Some of them understand why, and some do not, but they all understand that their happiness, the beauty of their city, the tenderness of their friendships, the health of their children, the wisdom of their scholars, the skill of their makers, even the abundance of their harvest and the kindly weathers of their skies, depend wholly on this child’s abominable misery.

The Ones Who Stay and Fight, by N.K. Jemisin:

But this is no awkward dystopia, where all are forced to conform. Adults who refuse to give up their childhood joys wear wings, too, though theirs tend to be more abstractly constructed. (Some are invisible.) And those who follow faiths which forbid the emulation of beasts, or those who simply do not want wings, need not wear them. They are all honored for this choice, as much as the soarers and flutterers themselves—for without contrasts, how does one appreciate the different forms that joy can take?

Why Don’t We Just Kill the Kid in the Omelas Hole, by Isabel J. Kim:

So they broke into the hole in the ground, and they killed the kid, and all the lights went out in Omelas: click, click, click. And the pipes burst and there was a sewage leak and the newscasters said there was a typhoon on the way, so they (a different “they,” these were the “they” in charge, the “they” who lived in the nice houses in Omelas [okay, every house in Omelas was a nice house, but these were Nice Houses]) got another kid and put it in the hole.

This post is part of February’s IndieWeb Carnival, in which Manuel Moreale prompts us to think about the various facets of digital relationships.

Our relationship to digital technology has been through a few different phases.

One: the census

In the first, computers were the realm of government and big business: vast databases that might be about us, but that we could never own or interrogate ourselves. Companies like IBM manufactured room-sized (and then cabinet-sized) machines that took a team of specialized technicians to operate. They were rare and a symbol of top-down power.

Punch cards were invented in the 1880s, and were machine-sortable even then, although not by anything we would recognize as a computer today. In the 1930s, a company called Dehomag, which was a 90%-owned subsidiary of IBM, used its punch card census technology to help the German Nazi party ethnically identify and sort the population. (Thomas Watson, IBM’s CEO at the time, even came to Germany to oversee the operation.)

The first general-purpose digital computer, ENIAC, was first put to use to determine the feasibility of the H bomb. Other mainframe computers were used by the US Navy for codebreaking, and by the US census bureau. By the sixties and seventies, though, they were commonplace in larger corporate offices and in universities for non-military, non-governmental applications.

Two: the desk

Personal computers decentralized computing power and put it in everybody’s hands. There was no overarching, always-on communications network for them to connect to, so every computer had its own copy of software that ran locally on it. There was no phoning home; no surveillance of our data; there were no ad-supported models. If you were lucky enough to have the not-insignificant sum of money needed to buy a computer, you could have one in your home. If you were lucky enough to have money left over for software, you could even do things with it.

The government and large institutions didn’t have a monopoly on computing power; theoretically, anyone could have it. Anyone could write a program, too, and (if you had yet more money to buy a modem) distribute it on bulletin board systems and online services. Your hardware was yours; your software was yours; once you’d paid your money, your relationship with the vendor was over.

For a while, you had a few options to connect with other people:

• Prodigy, an online service operated as a joint venture between CBS, IBM, and Sears
• CompuServe, which was owned and run by H&R Block
• America Online, which was originally a way for Atari 2600 owners to download new games and store high scores
• Independent bulletin boards, which were usually a single computer connected to a handful of direct phone lines for modems to connect to, run by an enthusiast

(My first after-school job was as a BBS system operator for Daily Information, a local information and classifieds sheet in my hometown.)

In 1992, in addition to bulletin board systems and online services, the internet was made commercially available. Whereas BBSes, AOL, etc were distinct walled gardens, any service that was connected to the internet could reach any other service. It changed everything. (In 1995, my BBS job expanded to running what became one of the first classifieds websites.)

But for a while, the decentralized, private nature of personal computing remained. For most private individuals, connecting to the internet was like visiting a PO box: you’d dial in, would upload and download any email you had pending, browse any websites you needed to, and then log off again. There was no way to constantly monitor people because internet users spent 23 hours of the day disconnected from the network.

Three: the cloud

Broadband, the iPhone, and wifi changed everything. Before the advent of broadband, most people needed to dial in to go online using their phone line. Before the iPhone, cell connections weren’t metered for data, and there was very little bandwidth to go around. Before wifi, a computer needed to physically be connected with a cable to go online.

With broadband and wifi, computers could be connected to the internet 24/7. With the iPhone, everyone had a computer in their pocket, that was permanently connected and could be constantly sending data back to online services — including your location and who was in your address book.

It was incredibly convenient and changed the world in hundreds of ways. The web in particular is a modern marvel; the iPhone is a feat of design and engineering. But what we lost was the decentralized self-ownership of our digital worlds. More than that, we lost an ability to be private that we’d had since the beginning of human civilization. It used to be that nobody needed to know where you were or what you were thinking about; that fundamental truth has gone the way of the dinosaur.

Almost immediately, our relationship to software changed in a few key ways:

• We could access all of our data from anywhere, on any device.
• Instead of buying a software package once, we were asked to subscribe to it.
• Instead of downloading or installing software, the main bulk of it could be run in a server farm somewhere.
• Every facet of our data was stored in one of these server farms.
• More data was produced about us as we used our devices — or even as we walked through our cities, shopped at stores, and met with other people — than we created intentionally ourselves.

While computing became infinitely easier to use and the internet became a force that changed global society in ways that I still believe are a net positive, surveilling us also became infinitely easier. Companies wanted to know exactly what we were likely to buy; politicians wanted to know how we might vote; law enforcement wanted to know if we were dangerous. All paid online services to build profiles about us that could be used to sell advertising, could be mined by the right buyer, and could even be used to influence elections.

Four: the farm

Our relationship is now changing again.

Whereas in the cloud era we were surveilled in order to profile us, our data is now being gathered for another set of reasons. We’re used to online services ingesting our words and actions in order to predict our behaviors and influence us in certain directions. We’re used to Target, for example, wanting to know if we’re pregnant so they can be the first to sell us baby gear. We’re not used to those services ingesting our words and actions in order to learn how to be us.

In our new relationship, software isn’t just set up to surveil us to report on us; it’s also set up to be able to do our work. GitHub Copilot learns from software we write so that it can write software automatically. Midjourney builds stunning illustrations and near-photorealistic images. Facebook is learning from the text and photos we upload so it can create its own text and realistic imagery (unlike many models, from data it actually has the license to). Far more than us being profiled, our modes of human expression are now being farmed for the benefit of people who hope to no longer have to hire us for our unique skills.

In the first era, technology was here to catalogue us.

In the second, it was here to empower us.

In the third, it was here to observe us.

In the fourth, it is here to replace us.

We had a very brief window, somewhere between the inception of the homebrew computer club and the introduction of the iPhone, where digital technology heralded distributed empowerment. Even then, empowerment was hardly evenly distributed, and any return to decentralization must be far more equitable than it ever was. But we find ourselves in a world where our true relationship is with power.

Of course, it’s a matter of degrees, and everything is a spectrum: there are plenty of services that don’tuse your data to train generative AI models, and there are plenty that don’t surveil you at all. There are also lots of applications and organizations that are actively designed to protect us from being watched and subjugated. New regulations are being proposed all the time that would guarantee our right to privacy and our right to not be included in training data.

Those might seem like technical decisions, but they’re really about preserving our ownership and autonomy, and returning those things to us when they’ve already been lost. They’re human, democratic decisions that seek to enforce a relationship where we’re in charge. They’re becoming more and more important every day.

I tried Arc Search, the new mobile app from the Browser Company. Its central insight is that almost every mobile browsing session starts with a web search; rather than giving you the usual list of results, it prioritizes building a web page for you that contains all the information you asked for. That way, the theory goes, you can see the information you need and be on your way faster. (You can still fall back to a Google search, although given that Google is going down the same path, that might not be as differentiated an experience — or as correct — as you might think.)

Obviously, I searched for myself. (Admit it, you would too.) Here’s what it gave me:

I have a few notes:

• This is a photo of my friend Tantek Çelik. I have a lot of respect for Tantek, and I think he’s done a lot to consistently work for the open web in a way that most of the rest of us haven’t always been able to. I also miss hanging out with him. But I am not him.
• I am not the Chief Technology Officer at The 19th. I was, but I haven’t been there for almost a year. Tyler Fisher is the CTO at The 19th, where he’s doing an excellent job.
• It’s not in the screenshot, but it also claims that I’m a contributor to The 19th. I wish I was a journalist of that calibre, but let’s be clear, I’m a web developer who blogs.

If this was an ordinary web search, you could easily see that pages that describe me as the CTO at The 19th are old, and that the photo is of Tantek. But it’s not an ordinary search at all, and Arc Search has presented the information as factual, without context or attribution, as a very simple website. There are three representative links presented further down the page, but in a way that is disconnected from the facts themselves.

I’m flattered by the comparison to Tantek, and while I’m not that keen on having my job position misrepresented, it’s not catastrophic. In other words, it could be worse. But consider if you weren’t just doing a vanity search, and instead were looking for something that actually mattered.

I’m also troubled by the role of training models themselves. When you remove attribution and display facts in this way, you give the appearance of objectivity without the requirement to actually be objective. It all comes down to which sources it checks and how the model is trained to report back — in other words, the biases of the developers.

If I search for “who should I follow in AI?” I get the usual AI influencers, with no mention of Timnit Gebru or Joy Buolamwini (who would be my first choices). If I ask who to follow in tech, I get Elon Musk. It undoubtedly has a lens through which it sees the world. That’s fine in itself — everyone does — but by removing context, you remove the clues that help you figure out what it is.

Finally, obviously, the sources themselves are automatically browsed by the app but don’t see the benefit of a human visitor. Real people sometimes pay to access content, or donate to support a nonprofit publisher, or watch an ad. Those things can be annoying but pay for the content to be produced in the first place. If we strip them away, there’s no writing, information, or expression for the app to summarize. A world where everyone uses an app like this is a death spiral to an information desert.

I guess what I’m saying is: thanks, I hate it. Give me context; give me nuance; give me the ability to think for myself. We built the world’s most incredible communication and knowledge-sharing medium, rich with diverse perspectives and alternative ideas; let’s not sanitize it through a banal filter that is designed to strip it of its humanity.