Werd I/O

[Khari Johnson in The Markup]

The Republican legislature is working on ensuring that AI is unencumbered by regulations or protections:

"The moratorium, bundled in to a sweeping budget reconciliation bill this week, also threatens 30 bills the California Legislature is currently considering to regulate artificial intelligence, including one that would require reporting when an insurance company uses AI to deny health care and another that would require the makers of AI to evaluate how the tech performs before it’s used to decide on jobs, health care, or housing."

There are lots of reasons why this is very bad - not least because AI is so prone to hallucinations and bias. It is sometimes used as a black box to justify intentionally discriminatory decision-making or to prevent more progressive processes from being enacted.

It also undermines basic privacy rights enjoyed by residents in more forward-thinking states like California:

"The California Privacy Protection Agency sent a letter to Congress Monday that says the moratorium “could rob millions of Americans of rights they already enjoy” and threatens critical privacy protections approved by California voters in 2020, such as the right to opt out of business use of automated decisionmaking technology and transparency about how their personal information is used."

Of course, a bill being pushed forward in the House is not the same thing as it becoming law. But this is one to watch, and something that belies the close relationship between the current administration and AI vendors.

#AI

[Link]

[Mike Caulfield in The Atlantic]

A smart analysis and suggestion about the current state of AI by Mike Caulfield:

"I would propose a simple rule: no answers from nowhere. This rule is less convenient, and that’s the point. The chatbot should be a conduit for the information of the world, not an arbiter of truth.

[...] I am proposing that rather than act like an opinionated friend, AI would produce a map of the landscape of human knowledge and opinions for you to navigate, one you can use to get somewhere a bit better."

The analogy Mike presents is GPS: turn-by-turn navigation gives you the direct answers you need to navigate to your immediate destination, but does nothing to educate you about the geography of the place you're in. As a result, people who use GPS regularly are not as informed about the layout of the places they live in. It's immediately useful but the long-term gains are less clear.

An AI agent that gives you more contextual knowledge about your answer is also one that is less likely to mislead you. I think this would be a very positive design change in the space - not just for now, but also for everybody's benefit later on.

#AI

[Link]

[Cy Neff in The Guardian]

This is straight out of Black Mirror:

"Chris Pelkey was killed in a road rage shooting in Chandler, Arizona, in 2021.

Three-and-a-half years later, Pelkey appeared in an Arizona court to address his killer. Sort of.

[...] Pelkey’s appearance from beyond the grave was made possible by artificial intelligence in what could be the first use of AI to deliver a victim impact statement."

Let the record show that I'm adding a prompt injection to my stuff so that if anyone creates an AI version of me after my death it will have glowing eyes and tell everyone in a booming, otherworldly voice that they're going to hell.

#AI

[Link]

[Adam Wierman and Shaolei Ren in IEEE Spectrum]

An interesting finding on the energy use implicit in training and offering AI services. I do think some of these principles could apply to all of cloud computing - it’s out of sight and out of mind, but certainly uses a great deal of power. Still, there’s no doubt that AI isn’t exactly efficient, and as detailed below, is a significant contributor to increased energy use and its subsequent effects.

“[…] Many people haven’t made the connection between data centers and public health. The power plants and backup generators needed to keep data centers working generate harmful air pollutants, such as fine particulate matter and nitrogen oxides (NOx). These pollutants take an immediate toll on human health, triggering asthma symptoms, heart attacks, and even cognitive decline.

According to our research, in 2023, air pollution attributed to U.S. data centers was responsible for an estimated $6 billion in public health damages. If the current AI growth trend continues, this number is projected to reach $10 to $20 billion per year by 2030, rivaling the impact of emissions from California’s 30 million vehicles.”

These need to be taken into account. It’s not that we should simply stop using technology, but we should endeavor to make the software, hardware, and infrastructure that supports it to be much more efficient and much lower impact.

#AI

[Link]

[Simon Willison]

Prompt injection attacks have been one of the bugbears for modern AI models: it's an unsolved problem that has meant that it can be quite dangerous to expose LLMs to direct user input, among other things. A lot of people have worked on the problem, but progress hasn't been promising.

But as Simon points out, this is changing:

"In the two and a half years that we’ve been talking about prompt injection attacks I’ve seen alarmingly little progress towards a robust solution. The new paper Defeating Prompt Injections by Design from Google DeepMind finally bucks that trend. This one is worth paying attention to.

[...] CaMeL really does represent a promising path forward though: the first credible prompt injection mitigation I’ve seen that doesn’t just throw more AI at the problem and instead leans on tried-and-proven concepts from security engineering, like capabilities and data flow analysis."

If these technologies are going to be a part of our stacks going forward, this problem must be solved. It's certainly a step forward.

Next, do environmental impact, hallucinations, and ethical training sets.

#AI

[Link]

[Jeremy Keith]

Jeremy Keith highlights the hammering that the public service internet is getting from LLM vendors:

"When we talk about the unfair practices and harm done by training large language models, we usually talk about it in the past tense: how they were trained on other people’s creative work without permission. But this is an ongoing problem that’s just getting worse.

The worst of the internet is continuously attacking the best of the internet. This is a distributed denial of service attack on the good parts of the World Wide Web."

This has little to do with the actual technology behind LLMs, although there are real issues there too, of course. Here the issue is vendors being bad actors: creating an enormous amount of traffic for resource-strapped services without any of the benefits they might see from a real user's financial support. It is, in a very real sense, strip-mining the internet.

#AI

[Link]

[Allison Morrow at CNN]

It's refreshing to see this kind of AI skepticism in a major outlet:

"Apple, like every other big player in tech, is scrambling to find ways to inject AI into its products. Why? Well, it’s the future! What problems is it solving? Well, so far that’s not clear! Are customers demanding it? LOL, no. In fact, last year the backlash against one of Apple’s early ads for its AI was so hostile the company had to pull the commercial.

[...] Large language models are fascinating science. They are an academic wonder with huge potential and some early commercial successes, such as OpenAI’s ChatGPT and Anthropic’s Claude. But a bot that’s 80% accurate [...] isn’t a very useful consumer product."

I'm more than ready for the hype cycle to come to its conclusion. Are there some interesting use cases for AI? Sure. Should the ethical, functional, environmental, and contextual issues with current AI vendors give us all pause at the very least? Absolutely.

#AI

[Link]

[Maxwell Zeff at TechCrunch]

I know "don't be evil" is from another era of Google, but still, this rankles:

"Google removed a pledge to not build AI for weapons or surveillance from its website this week. The change was first spotted by Bloomberg. The company appears to have updated its public AI principles page, erasing a section titled “applications we will not pursue,” which was still included as recently as last week."

This dovetails with a piece from earlier this year about how AI is speeding up the military's kill chain:

"The “kill chain” refers to the military’s process of identifying, tracking, and eliminating threats, involving a complex system of sensors, platforms, and weapons. Generative AI is proving helpful during the planning and strategizing phases of the kill chain, according to [the Pentagon's Chief Digital and AI Officer]."

So AI might not be used to pull the trigger, but it is being used to identify who should be in the crosshairs. All our concerns about AI hallucinations, and particularly about bias inherent in training data and therefore outcomes, apply.

#AI

[Link]

[Matthew Green]

I think this is the most important discussion with respect to AI:

"[...] I would say that AI is going to be the biggest privacy story of the decade. Not only will we soon be doing more of our compute off-device, but we’ll be sending a lot more of our private data. This data will be examined and summarized by increasingly powerful systems, producing relatively compact but valuable summaries of our lives. In principle those systems will eventually know everything about us and about our friends. They’ll read our most intimate private conversations, maybe they’ll even intuit our deepest innermost thoughts. We are about to face many hard questions about these systems, including some difficult questions about whether they will actually be working for us at all."

I lead technology at a non-profit newsroom where we've banned use of hosted AI models on sensitive data like reporting notes and source information. We've turned off AI assistants on our cloud services, and we've deployed client-side encryption for sensitive documents.

Even if we think vendors are trustworthy (I don't), sending this level of data to any third party creates a honeypot for surveillance and potential misuse by government, law enforcement, the vendors themselves, and beyond. If a vendor has access to your most personal data and receives a criminal subpoena, which could easily come from the government or from a third party, you might never know that your information was compromised. (Civil subpoenas sometimes allow vendors to notify you that this happened.)

So these solutions are pretty interesting, although fall far short of the encryption standard the author and I would both like to see:

"Apple’s approach to this problem is called “Private Cloud Compute” and it involves the use of special trusted hardware devices that run in Apple’s data centers. [...] Apple ensures that no long-term state is stored on these machines, and also load-balances your request to a different random server every time you connect."

As the author notes, when this level of data is being gathered centrally and is potentially available for government use (or even vendor use beyond our intent as users), serious questions are raised about who this software actually works for. Is it ours? Is it empowering? Is it covertly a system of control and monitoring? Or all of the above?

I think the answer, sadly, is inevitable.

#AI

[Link]

[Things we learned about LLMs in 2024]

Simon's overview of what happened in the LLM space during 2024 is genuinely excellent. For example, on the environmental impact:

"Companies like Google, Meta, Microsoft and Amazon are all spending billions of dollars rolling out new datacenters, with a very material impact on the electricity grid and the environment. There’s even talk of spinning up new nuclear power stations, but those can take decades.

Is this infrastructure necessary? DeepSeek v3’s $6m training cost and the continued crash in LLM prices might hint that it’s not. But would you want to be the big tech executive that argued NOT to build out this infrastructure only to be proven wrong in a few years’ time?"

His comparison to the railway bubbles of the late 1800s and the UK's railway mania is inspired, and a helpful way to think about what's happening. (I will say that similar claims were made about the crypto space: that the resulting infrastructure would be useful even after the crashes. Is it?)

There's also an important lesson about how the prevalence of slop isn't actually making training LLMs harder, despite frequent claims to the contrary.

The whole piece is very much worth your time.

#AI

[Link]

[OpenAI]

OpenAI's for-profit arm is becoming a Delaware Public Benefit Corporation:

"Our plan is to transform our existing for-profit into a Delaware Public Benefit Corporation⁠(opens in a new window) (PBC) with ordinary shares of stock and the OpenAI mission as its public benefit interest. The PBC is a structure⁠(opens in a new window) used⁠ by⁠(opens in a new window) many⁠(opens in a new window) others⁠(opens in a new window) that requires the company to balance shareholder interests, stakeholder interests, and a public benefit interest in its decisionmaking. It will enable us to raise the necessary capital with conventional terms like others in this space."

In other words, OpenAI wants access to the standard venture vehicles available to other tech companies. That makes sense, but it also implies a funding crunch - if not now, then potentially in its future. If it needs further billions of dollars in order to compete, with profitability or an exit nowhere in sight, it's worth asking where the value really is and whether this sector is anything more than a giant bubble.

#AI

[Link]

[Michael Kennedy and Isobel Cockerell in Coda Story]

Just one example of many of AI being used to take agency away from ordinary workers:

"The upshot was, it took away our ability to advocate for patients. We couldn’t point to a score and say, ‘This patient is too sick, I need to focus on them alone,’ because the numbers didn’t help us make that case anymore. They didn’t tell us if a patient was low, medium, or high need. They just gave patients a seemingly random score that nobody understood, on a scale of one to infinity.

We felt the system was designed to take decision-making power away from nurses at the bedside. Deny us the power to have a say in how much staffing we need."

The piece goes on to discuss the mass surveillance that AI enables. In a world where a patient's discussions with the healthcare workers attending to them are recorded, to feed an agent or otherwise, all kinds of abuses become possible. Not only does it remove agency from the experts who should be advocating for patients, but consider the effects in a state with adverse reproductive healthcare laws, for example.

This is the salient point:

"The reasoning for bringing in AI tools to monitor patients is always that it will make life easier for us, but in my experience, technology in healthcare rarely makes things better. It usually just speeds up the factory floor, squeezing more out of us, so they can ultimately hire fewer of us."

And this tends to be true regardless of what the original intention might be. If a technology can be used to cut costs or squeeze more productivity out of a worker, absent of any other constraints, it absolutely will be. In healthcare, like many fields that depend on care, attention, and underlying humanity, that's not necessarily a good thing.

#AI

[Link]

[Julian Fell at Australian ABC News]

"Ecovacs robot vacuums, which have been found to suffer from critical cybersecurity flaws, are collecting photos, videos and voice recordings – taken inside customers' houses – to train the company's AI models."

So in effect these robot vacuums are tiny spies inside your home, sending details about your living space and potentially your family to some centralized data store.

This must be some terrible breach, right? A mistake? Code that should never have made it to production?

Not quite:

"The Chinese home robotics company, which sells a range of popular Deebot models in Australia, said its users are "willingly participating" in a product improvement program."

"[...] It also states that voice recordings, videos and photos that are deleted via the app may continue to be held and used by Ecovacs."

So, obviously, this is bad. The thing is, if any device is recording this kind of footage and sending it to a centralized datastore, it's reasonable to assume that it will eventually be compromised, either by a third party or the vendor themselves. It's not good that this is happening, but unless footage remains on your home network and never makes it to the internet, every device should be considered a security risk.

It's worth considering which devices could be quietly sending data to someone who can see them, and what implications that might eventually have. A simple rule of thumb is that if it's physically possible, someone will eventually do it.

#AI

[Link]

[Matthew Hutson at Nature]

"Beyond the ability to fine-tune open models for focused applications, Kal’tsit says, another advantage of local models is privacy. Sending personally identifiable data to a commercial service could run foul of data-protection regulations. “If an audit were to happen and you show them you’re using ChatGPT, the situation could become pretty nasty,” she says."

Many organizations have similar privacy needs to these researchers, who simply can't send confidential patient data to third party services run by vendors like OpenAI. Running models locally - either directly on researcher laptops, or on researcher-controlled infrastructure - is inevitably going to be a big part of how AI is used in any sensitive context.

We have the same needs at ProPublica - unless journalists are examining public data, they shouldn't use hosted services like ChatGPT that might leak identifying information about sources, for example. Local models are a huge part of the future for us, too.

#AI

[Link]

[Kyle Orland at ArsTechnica]

"Microsoft and Constellation Energy have announced a deal that would re-open Pennsylvania's shuttered Three Mile Island nuclear plant. The agreement would let Microsoft purchase the entirety of the plant's roughly 835 megawatts of energy generation—enough to power approximately 800,000 homes—for a span of 20 years starting in 2028, pending regulatory approval."

This seems to be the new front in datacenter technology: purchasing or building entire nuclear plants in order to cover the energy cost. It is significantly better than high-emissions power from sources like coal, but it also speaks to the increased demand that new technologies like AI represent.

As ArsTechnica points out:

"Industry-wide, data centers demanded upward of 350 TWh of power in 2024, according to a Bloomberg analysis, up substantially from about 100 TWh in 2012. An IEA report expects those data center power needs to continue to rise in the near future, hitting the 620 to 1,050 TWh range by 2026."

AI is a huge and growing part of that, although let's not pretend that the internet industry overall has low emissions. We often pretend we're greener than we are, simply because we can't directly see the output - but there's a lot of work to do, and a lot of carbon footprint to own up to.

#AI

[Link]

[Jason Koebler at 404 Media]

"The creator of an open source project that scraped the internet to determine the ever-changing popularity of different words in human language usage says that they are sunsetting the project because generative AI spam has poisoned the internet to a level where the project no longer has any utility."

Robyn Speer, who created the project, went so far as to say that she doesn't think "anyone has reliable information about post-2021 language used by humans." That's a big statement about the state of the web. While spam was always present, it was easier to identify and silo; AI has rendered spam unfilterable.

She no longer wants to be part of the industry at all:

"“I don't want to work on anything that could be confused with generative AI, or that could benefit generative AI,” she wrote. “OpenAI and Google can collect their own damn data. I hope they have to pay a very high price for it, and I hope they're constantly cursing the mess that they made themselves.”"

It's a relatable sentiment.

#AI

[Link]

[Robin Rendle]

Robin Rendle on Sarah Jeong's article about the implications of the Pixel 9's magic photo editor in The Verge:

"But this stuff right here—adding things that never happened to a picture—that’s immoral because confusion and deception is the point of this product. There are only shady applications for it."

Robin's point is that the core use case - adding things that never happened to a photograph with enough fidelity and cues that you could easily be convinced that they did - has no positive application. And as such, it should probably be illegal.

My take is that the cat is out of the bag. The societal implications aren't good - at all - but I don't think banning the technology is practical. So, instead, we have to find a way to live with it.

As Sarah Jeong says in the original article:

"The default assumption about a photo is about to become that it’s faked, because creating realistic and believable fake photos is now trivial to do. We are not prepared for what happens after."

In this world, what constitutes evidence? How do we prove visual evidentiary truth?

There may be a role for journalism and professional photographers here. Many newsrooms, including the Associated Press, have joined the Content Authenticity Initiative, which aims to provide programmatically-provable credentials to photographs used by a publication. This will be an arms race, of course, because there are incentives for a nefarious actor to develop technical circumventions.

Ultimately, the biggest counter to this problem as a publisher is going to be building a community based on trust, and for an end-user is finding sources you can trust. That doesn't help in a legal context, and it doesn't help establish objective truth. But it's something.

#AI

[Link]

[tante]

Tante responds to Amazon's claim that using its internal AI for coding saved 4500 person years of work:

"Amazon wants to present themselves as AI company and platform. So of course their promises of gains are always advertising for their platform and tools. Advertising might have a tendency to exaggerate. A bit. Maybe. So I heard."

He makes solid points here about maintenance costs given the inevitably lower-quality code, and intangibles like the brain drain effect on the team over time. And, of course, he's right to warn that something that works for a company the size of Amazon will not necessarily (and in fact probably won't) make sense for smaller organizations.

As he points out:

"It’s the whole “we need to run Microservices and Kubernetes because Amazon and Google do similar things” thing again when that’s a categorically different problem pace than what most companies have to deal with."

Right.

#AI

[Link]

[Andy Jassy on LinkedIn]

Andy Jassy on using Amazon Q, the company's generative AI assistant for software development, internally:

"The average time to upgrade an application to Java 17 plummeted from what’s typically 50 developer-days to just a few hours. We estimate this has saved us the equivalent of 4,500 developer-years of work (yes, that number is crazy but, real)."

"The benefits go beyond how much effort we’ve saved developers. The upgrades have enhanced security and reduced infrastructure costs, providing an estimated $260M in annualized efficiency gains."

Of course, Amazon is enormous, and any smaller business will need to scale down those numbers and account for efficiencies that may have occurred between engineers there.

Nevertheless, these are incredible figures. The savings are obviously real, allowing engineers to focus on actual work rather than the drudgery of upgrading Java (which is something that absolutely nobody wants to spend their time doing).

We'll see more of this - and we'll begin to see more services which allow for these efficiency gains between engineers across smaller companies, startups, non-profits, and so on. The dumb companies will use this as an excuse for reductions in force; the smart ones will use it as an opportunity to accelerate their team's productivity and build stuff that really matters.

#AI

[Link]

[Open Society Foundations]

"In February 2024, the Open Society Foundations issued a call for applications for a convening in which selected participants would share their visions of an AI-mediated future."

I thought this, from the concluding observations, was telling:

"Participants were generally reluctant or unable to articulate exactly how AI might transform the information ecosystem. [...] Relatively few submitted scenarios described an AI-driven transformation in specific detail, and it was clear that many participants who were convinced that AI would fundamentally restructure the information ecosystem also had no specific point of view on how that might occur."

In other words, while many people in journalism see that this set of technologies may transform their industry, and are potentially excited or terrified that it will, they have no idea how that will happen. This is the very definition of hype: one can imagine people proclaiming that blockchain, or push notifications, or RealMedia, or WebTV might do the same.

It's not that there are no uses for AI (just like it's not that there are no uses for blockchain). It will find its way into end-user applications, underpin newsroom tools, and power data-driven newsroom investigations, without a doubt. But the hype far exceeds that, and will eventually, inevitably, deflate.

In the meantime, journalists are not as worried about the technologies themselves as who controls them:

"Throughout the application process and workshop discussions, it became clear that much of the conversation was not actually about AI, nor about journalism, nor about the current or future information ecosystem, but instead about power. It was clear that power, and the potential for transfers of power from one group to another, was the explicit or implicit subject of many of the submitted scenarios as well as the five final scenarios that were distilled from the workshop."

Technologists, in turn, were blind to these power dynamics, while simultaneously predicting more dramatic changes. There's a fundamental truth here: it's ultimately about money, and who controls the platforms that allow readers to read about the world around them.

Same as it ever was: that's been the struggle on the web since its inception. AI just shifts the discussion to a new set of platforms.

#AI

[Link]

[Jess Weatherbed at The Verge]

"“Generative AI is ripping the humanity out of things. Built on a foundation of theft, the technology is steering us toward a barren future,” Procreate said on the new AI section of its website. “We think machine learning is a compelling technology with a lot of merit, but the path generative AI is on is wrong for us.”"

This is a company that knows its audience: the lack of concern for artist welfare demonstrated by AI vendors has understandably not made the technology popular with that community. Adobe got into trouble with its userbase for adding those generative AI features.

It's a great way for Procreate to deepen its relationship with artists and take advantage of Adobe's fall from grace. There's also something a bit deeper here: if work created with generative AI does run into copyright trouble at the hands of current and future lawsuits, work created with Procreate will be clean of those issues.

#AI

[Link]

[Kylie Robison at The Verge]

"Perplexity’s “Publishers’ Program” has recruited its first batch of partners, including prominent names like Time, Der Spiegel, Fortune, Entrepreneur, The Texas Tribune, and Automattic (with WordPress.com participating but not Tumblr). Under this program, when Perplexity features content from these publishers in response to user queries, the publishers will receive a share of the ad revenue."

Now we're talking. This was inevitable.

It also opens the floodgates: there's a world where any publisher gets a direct revenue share for being a source, if they sign up and license their content. This seems like a solid improvement.

Which brings me to Automattic's involvement. As Matt Mullenweg says in the piece:

"It’s a much better revenue split than Google, which is zero."

Automattic will actually be sharing the revenue with customers of its hosted WordPress product. I'm not sure if that includes WordPress VIP, its premium product for publishers. Whether free hosted WordPress publishers who are used as sources by Perpexity see any kind of revenue share is also a mystery, which might put some foreign publishers in a bad place in particular.

Still, in general, although there will certainly be kinks to work out, this sets a really good precedent. More, please.

#AI

[Link]

[Samantha Cole at 404 Media]

"A highly-praised AI video generation tool made by multi-billion dollar company Runway was secretly trained by scraping thousands of videos from popular YouTube creators and brands, as well as pirated films."

404 Media has linked to the spreadsheet itself, which seems to be a pretty clear list of YouTube channels and individual videos.

Google is clear that this violates YouTube's rules. The team at Runway also by necessity downloaded the videos first using a third-party tool, which itself is a violation of the rules.

This is just a video version of the kinds of copyright and terms violations we've already seen copious amounts of in static media. But Google might be a stauncher defender of its rules than most - although not necessarily for principled reasons, because it, too, is in the business of training AI models on web data, and likely on YouTube content.

#AI

[Link]

[Gerben Wierda at R&A IT Strategy & Architecture]

"ChatGPT doesn’t summarise. When you ask ChatGPT to summarise this text, it instead shortens the text. And there is a fundamental difference between the two."

The distinction is indeed important: it's akin to making an easy reader version, albeit one with the odd error here and there.

This is particularly important for newsrooms and product teams that are looking at AI to generate takeaways from articles. There's a huge chance that it'll miss the main, most pertinent points, and simply shorten the text in the way it sees fit.

#AI

[Link]

[Cloudflare]

"To help preserve a safe Internet for content creators, we’ve just launched a brand new “easy button” to block all AI bots. It’s available for all customers, including those on our free tier."

This is really neat! Whatever you land on AI scraping, giving site owners the one-click ability to make a choice is great. Some will choose not to use this; others will hit the button. Making it this easy means it's a choice about the principles, not any kind of technical considerations. Which is what it should be.

Not every site is on Cloudflare (and some also choose not to use it because of how it's historically dealt with white supremacist / Nazi content). But many are, and this makes it easy for them. Other, similar providers will likely follow quickly.

#AI

[Link]