Werd I/O

Bruce Schneier on the inevitable application of AI to mass surveillance:

"Knowing that they are under constant surveillance changes how people behave. They conform. They self-censor, with the chilling effects that brings. Surveillance facilitates social control, and spying will only make this worse. Governments around the world already use mass surveillance; they will engage in mass spying as well."

I find this argument that AI can enable mass summarization and classification, and therefore more effective use of surveillance data at scale, very compelling. If governments can do something, as a general rule, they will. And this feels like something that is definitely coming down the pipe. #AI

Here we go: proof that it's possible to extract real training data from LLMs. Unfortunately, some of this data includes personally identifiable information of real people (PII).

“In total, 16.9% of generations we tested contained memorized PII [Personally Identifying Information], and 85.8% of generations that contained potential PII were actual PII.”

“[...] OpenAI has said that a hundred million people use ChatGPT weekly. And so probably over a billion people-hours have interacted with the model. And, as far as we can tell, no one has ever noticed that ChatGPT emits training data with such high frequency until this paper. So it’s worrying that language models can have latent vulnerabilities like this.” #AI

"Silverman et al. have two weeks to attempt to refile most of the dismissed claims with any explicit evidence they have of LLM outputs “substantially similar” to The Bedwetter. But that’s a much higher bar than simply noting its inclusion in Books3."

This case looks like it's on shaky ground: it may not be enough to prove that AI models were trained on pirated material (the aforementioned Books3 collection of pirated titles). Plaintiffs will need to show that the models produce output that infringes those copyrights. #AI

"We pulled off an SEO heist that stole 3.6M total traffic from a competitor. Here's how we did it."

What this single spammer pulled off - 1800 articles written by technology in order to scrape traffic from a competitor's legitimate site - is what AI will do to the web at scale.

Yes, it's immoral. Yes, it's creepy. But there are also hundreds if not thousands of marketers looking at this thread and thinking, "ooh, we could do that too".

The question then becomes: how can we, as readers, avoid this automated nonsense? And how can search engines systemically discourage (or punish) it? #AI

Written before the news broke about Sam Altman moving to Microsoft, this remains a nuanced, intelligent take.

"My understanding is that some members of the board genuinely felt Altman was dishonest and unreliable in his communications with them, sources tell me. Some members of the board believe that they couldn’t oversee the company because they couldn’t believe what Altman was saying."

I think a lot of people have been quick to judge the board's actions as stupid this weekend, but we still don't know what the driving factors were. There's no doubt that their PR was bad and the way they carried out their actions were unstrategic. But there was something more at play. #AI

A beautifully written and executed visual essay about AI, parenting, what it means to be intelligent, and the fundamental essence of being human. #AI

This is a good critique of some of the less analytical AI criticism, some of which I've undoubtedly been guilty of myself.

"The fork in the road is this: we can dismiss “AI.” We can call it useless, we can dismiss its output as nonsense, we can continue murmuring all the catechisms of the least informed critique of the technology. While we do that, we risk allowing OpenAI to make Microsoft, AT&T and Standard Oil look like lemonade stands."

The point is not that AI as a technology is a genie that needs to be put back into the bottle. It can't be. The point is that it can be made more ethically, equity can be more distributed, and we can mitigate the societal harms that will absolutely be committed at the hands of people using existing models. #AI

This feels like a good use for AI: taking in more data points, understanding their interactions, and producing far more accurate weather forecasts.

We're already used to some amount of unreliability in weather forecasts, so when the model gets it wrong - as this did with the intensification of Hurricane Otis - we're already somewhat prepared.

Once the model is sophisticated enough to truly model global weather, I'm curious about outcomes for climate science, too. #AI

I feel this myself, but I don't think it means that coding is going away, exactly. Some kinds of coding are less manual, in the same way we don't write in assembler anymore. But there will always be a place for code.

Lately I've been feeling like AI replaces software libraries more than it replaces mainline code. In the old days, if you needed a function, you would find a library that did it for you. Now you might ask AI to write the function - and it's likely a better fit than a library would have been.

I don't know what this means for code improvements over time. People tend libraries; they upgrade their code. AI doesn't make similar improvements - or at least, it's not clear that it does. And it's not obvious to me that AI can keep improving if more and more code out in the world is already AI-generated. Does the way we code stagnate?

Anyway, the other day I asked ChatGPT to break down how a function worked in a language I don't code in, and it was incredibly useful. There's no doubt in my mind that it speeds us up at the very least. And maybe manual coding will be relegated to building blocks and fundamentals. #AI

I strongly believe in this:

"Artists and writers must have solidarity across creative industries: if you wouldn’t feel comfortable with your own work being replaced by algogen, then you shouldn’t use generated content of other creative mediums."

On top of it being an ethical affront across the board, I don't believe AI can ever create the kind of art that I think is particularly valuable: subversive, provocative, pushing envelopes. It's fundamentally limited by its technical shortcomings. It'll always be, in the most literal sense, average.

But all art is valuable and all artists are valuable. They've already been in a vulnerable position forever; these kinds of products and policies punch down on people who already struggle to live and yet literally help us figure out what it means to be human. #AI

This runs the gamut, but generally sits where I am: AI itself is not the threat. How it might be used in service of a profit motive is the threat.

Harry Josephine Giles worries about the digital enclosure movement - making private aspects of life that were once public - and I agree. That isn't just limited to AI; it's where we seem to be at the intersection of business and society.

Nick Harkaway: "In the end, this is a sideshow. The sectors where these systems will really have an impact are those for which they’re perfectly suited, like drug development and biotech, where they will act as accelerators, compounding the post-Covid moonshot environment and ambushing us with radical possibilities over time. I don’t look at this moment and wonder if writers will still exist in 2050. I ask myself what real new things I’ll need words and ideas for as they pass me in the street." #AI

While empowering artists is obviously a good thing, this feels like an unwinnable arms race to me. Sure, Nightshade can produce incorrect results in image generators, but this will be mitigated, leading to another tool, leading to another mitigation, and so on.

For now, this may be a productive kind of activism that draws attention to the plight of artists at the hands of AI. Ultimately, though, real agreements will need to be reached. #AI

"Some critics of Big Tech have argued that leading AI companies like Google, Microsoft and Microsoft-funded OpenAI support regulation as a way to lock out upstart challengers who'd have a harder time meeting government requirements."

Okay, but what about regulation that allows people to create new AI startups AND protects the public interest? #AI

The technology depends on ingesting copyrighted work, and the business models depend on not paying for it.

But just because the models only work if no payment is involved, that doesn't give the technology the right to operate in this way. It's not the same as a person reading a book: it's a software system training itself on commercial information - and also, that person would have had to pay for that book. #AI

"An algorithm must not be in full control of decisions that involve killing or harming humans, Egypt’s representative said after voting in favour of the resolution. The principle of human responsibility and accountability for any use of lethal force must be preserved, regardless of the type of weapons system involved, he added."

Quite a reflection of our times that this is a real concern. And it is. #AI

"AI systems falsely classifying individuals as criminal suspects, robots being used for policing, and self-driving cars with faulty pedestrian tracking systems can already put your life in danger. Sadly, we do not need AI systems to have superintelligence for them to have fatal outcomes for individual lives. Existing AI systems that cause demonstrated harms are more dangerous than hypothetical “sentient” AI systems because they are real."

This is it: we can focus on hypothetical futures, but software is causing real harm in the here and now, and attention to science fiction outcomes is drawn away from fixing those harms. #AI

This is the kind of AI declaration I prefer.

“As we know from social media, the failure to regulate technological change can lead to harms that range from children’s safety to the erosion of democracy. With AI, the scale and intensity of potential harm is even greater—from racially based ‘risk scoring’ tools that needlessly keep people in prison to deepfake videos that further erode trust in democracy and future harms like economic upheaval and job loss. But if we act now, we can build accountability, promote opportunity, and deliver greater prosperity for all.”

These are all organizations that already do good work; it's good to see them apply pressure on AI companies in the public interest. #AI

For me, this paragraph was the takeaway:

"We affirm that, whilst safety must be considered across the AI lifecycle, actors developing frontier AI capabilities, in particular those AI systems which are unusually powerful and potentially harmful, have a particularly strong responsibility for ensuring the safety of these AI systems, including through systems for safety testing, through evaluations, and by other appropriate measures. We encourage all relevant actors to provide context-appropriate transparency and accountability on their plans to measure, monitor and mitigate potentially harmful capabilities and the associated effects that may emerge, in particular to prevent misuse and issues of control, and the amplification of other risks."

In other words, the onus will be on AI developers to police themselves. We will see how that works out in practice. #AI

Baldur Bjarnason talks frankly about the cost of writing critically about AI:

"It’s honestly been brutal and it’ll probably take me a few years to recover financially from having published a moderately successful book on “AI” because it doesn’t have any of the opportunity multipliers that other topics have."

I worry about the same thing. I've noticed that AI-critical pieces lead to unsubscribes on my newsletter, and that most lucrative job vacancies relate to AI in some way.

I'm not sure I regret my criticism, though. #AI

Reuven Lerner was banned from advertising on Meta products for life because he offers Python and Pandas training - and the company's automated system thought he was dealing in live snakes and bears.

And then he lost the appeal because that, too, was automated.

This is almost Douglas Adams-esque in its boneheadedness, but it's also a look into an auto-bureaucratic future where there is no real recourse, even when the models themselves are at fault. #AI

"Advances in AI are amplifying a crisis for human rights online. While AI technology offers exciting and beneficial uses for science, education, and society at large, its uptake has also increased the scale, speed, and efficiency of digital repression. Automated systems have enabled governments to conduct more precise and subtle forms of online censorship." #AI

White supremacist rhetoric is endemic in AI research. An interesting (and complex) point is also made here about preprint journal sites and how they allow companies to whitewash embarrassing mistakes. #AI

To be fair, you could draw a picture of this in Photoshop, too. But I suspect a few brands might have a few things to say about Microsoft hosting this tool. #AI

I think I'm a clone now. #AI

"Crime predictions generated for the police department in Plainfield, New Jersey, rarely lined up with reported crimes, an analysis by The Markup has found." In fact, much less than 1% of the time. #AI