Dustin Curtis, the creator of Svbtle, recently wrote:
Cole Peters responded:
I would go further than this. What Dustin seems to have been talking about was a kind of data-driven user research: getting feedback from users in aggregate.
This is realistically useful for a small subset of the tasks involved with building a great product.
User research is key to building great products. (By the way, we open-sourced our user research materials.) But deliberate research is far more useful than collecting aggregate data about user details, and reducing your userbase to a series of statistics.
Certainly, keeping track of key performance indicators about your platform can help you understand how it's doing. If nobody's posting to a social network after a month, you've got a problem, for example. But I'd argue that building and improving your tools - i.e., actually pushing forwards - requires a more humanist approach.
You need to talk to people. A lot of people. You need to ask the right questions, but mostly you need to listen to them, and understand as well as possible not just the needs they're telling you, but also their unspoken needs. The things they reveal as insights in conversation. Similarly, you can watch them as they use your product, and even go so far as to track their eye movements, individual clicks, and tiny physical responses.
These conversations are often compensated, and they always happen with the full consent of the user. There's nothing hidden or nefarious about them, and they are not asked to reveal any personal information that they don't want to. They're also far more useful than bulk information that's totally disconnected from the individual human context of the user.
What aggregate statistics are useful for: demographic information for targeted advertising. Let's not trick ourselves into thinking that the assumptions that make ads possible are absolute, unmovable, or necessary to build any kind of well-designed product.
I’m writing about the intersection of the internet, media, and society. Sign up to my newsletter to receive every post and a weekly digest of the most important stories from around the web.