If I was a nefarious software company, here’s how I might be thinking:

AI functionality tends to require that data is sent to a centralized service for processing.

This is often data that is not being shared online in any other way that is easily available for analysis: existential work questions, internal documents, and so on.

This makes it very valuable data to sell to brokers or to use in targeting advertising.

So, let’s add lots of AI functionality to our services to encourage people to share that data with us.

We’ll provide AI services.

We’ll mine the data that is provided to us when people use those AI services.

And then we’ll sell it.

The AI revolution is also the private data sharing revolution. It’s worth asking: does this AI feature I’m interested in using puncture a privacy hole in the service it is a part of? Who am I sharing this information with? What will they do with it?