When the news broke that Facebook experimented on users' emotions by manipulating their feeds, one of the most common defenses of Facebook that I heard was, "But all websites experiment on their users!"
Yesterday, we saw Christian Rudder at OkCupid, rather colorfully, make this facile argument. OkCupid has a history of experimenting with its matching algorithm and selections (and hence with users).
"But guess what, everybody: if you use the internet, you're the subject of hundreds of experiments at any given time, on every site. That's how websites work," says Rudder.
And despite his finger-wagging, dick-like phrasing that only entrepreneurs can achieve, Rudder is correct to a point.
Yesterday, we saw Christian Rudder at OkCupid, rather colorfully, make this facile argument. OkCupid has a history of experimenting with its matching algorithm and selections (and hence with users).
"But guess what, everybody: if you use the internet, you're the subject of hundreds of experiments at any given time, on every site. That's how websites work," says Rudder.
And despite his finger-wagging, dick-like phrasing that only entrepreneurs can achieve, Rudder is correct to a point.
Any website run by people who care even a little will conduct experiments on its users. The most common form of experiment would be the A/B test. Indeed, whole companies have cropped up to help websites create multiple versions of their product, see which version evokes the best response from users (measured in online engagement, user sign-ups, e-commerce revenue, etc.), and choose the winning design as the final version of the product before repeating the cycle all over again.
People A/B test all the time. For example, the Obama campaign streamlined the donation process via a series of experiments on their donors. And that was for the 2008 election! If this kind of experiment were such a scandal, surely political opponents would have jumped on it.
But people didn't get upset about the Obama campaign's test, or indeed, the vast majority of online experiments. Rudder is right.
That, however, doesn't mean that everyone upset by Facebook is wrong. Because not all experiments are the same. And it comes down to user expectations and incentives.
People A/B test all the time. For example, the Obama campaign streamlined the donation process via a series of experiments on their donors. And that was for the 2008 election! If this kind of experiment were such a scandal, surely political opponents would have jumped on it.
But people didn't get upset about the Obama campaign's test, or indeed, the vast majority of online experiments. Rudder is right.
That, however, doesn't mean that everyone upset by Facebook is wrong. Because not all experiments are the same. And it comes down to user expectations and incentives.
The Love Fairy In the case of a dating site like OkCupid, users sign up for the product because of the matching algorithm. They understand that a company, not the love fairy, is assigning matches for users. And it's reasonable to assume that the dating site would continue to play with and tweak its matching algorithm, ostensibly for the purpose of improving it. After all, what's a dating site but a user interface slapped on a matching algorithm with some critical mass of users?
So, while Rudder is being intentionally flippant in his blog post, what OkCupid has done is rather in line with user expectations.
What Facebook did however is not. Why? Because users expect that their feeds are more or less natural representations of their friends' worlds. That's what Facebook pushes. When Facebook released their 10th anniversary "look back" videos, the point was, "Hey, your life and your friends' lives are documented here in a beautiful, pure, holistic, almost documentary way." But then when users learned that their emotions had been toyed with, suddenly, their expectations were defied.
Sure, it was an experiment, but the experiment ran contrary to what people assumed was the purpose of the product.
And this leads us to incentives. Facebook's experiment, i.e. to prove that people's emotions can be compromised (to tailor your emotional state for an ad rather than to target you with an ad), in actuality did not run contrary to the purpose of the product. Because the purpose of the product is not to allow people to experience life online or whatever vague, bubbly BS the look back videos would have users believe. The purpose of Facebook is to display ads to its users, using its vast dataset to improve the reach and effectiveness of those ads.
Through Facebook's experiment, people realized that they were using a product owned by a company whose incentives were vastly different from their own. And this discordance was uncomfortable. The experiment signaled the direction in which Facebook might permanently move -- a direction in which the already artificial experience of life via an EdgeRank-directed feed becomes more artificial as Facebook plays with that feed for the purpose of aligning your emotional state to its ad content.
On the flip side, whether OkCupid admits it or not, their incentives (and hence their experiments) are aligned with user goals. A dating site that advertises itself as one that maintains a subpar matching algorithm is a site that'll likely go out of business quickly. The company is incentivized ultimately to improve their matching algorithm, even if that means making it worse for a few users in the short term in order to run an experiment. Sure, some of OkCupid's experiments seem audacious, but the purpose was to better understand the factors that lead to successful matches on the dating website. And users get that.
I don't think most folks would have all websites cease testing their products on their users. That iterative improvement on user feedback is invaluable. Instead, what companies should consider is whether their tests are in line with user expectations and whether those tests serve to improve the user experience of the product. If there's a disconnect, then that should give a company pause.
This article is an experiment. Leave your comments in the comment section, and if you're not a bot trying to sell weight loss pills, who knows, maybe I'll change what I say based on your feedback.
So, while Rudder is being intentionally flippant in his blog post, what OkCupid has done is rather in line with user expectations.
What Facebook did however is not. Why? Because users expect that their feeds are more or less natural representations of their friends' worlds. That's what Facebook pushes. When Facebook released their 10th anniversary "look back" videos, the point was, "Hey, your life and your friends' lives are documented here in a beautiful, pure, holistic, almost documentary way." But then when users learned that their emotions had been toyed with, suddenly, their expectations were defied.
Sure, it was an experiment, but the experiment ran contrary to what people assumed was the purpose of the product.
And this leads us to incentives. Facebook's experiment, i.e. to prove that people's emotions can be compromised (to tailor your emotional state for an ad rather than to target you with an ad), in actuality did not run contrary to the purpose of the product. Because the purpose of the product is not to allow people to experience life online or whatever vague, bubbly BS the look back videos would have users believe. The purpose of Facebook is to display ads to its users, using its vast dataset to improve the reach and effectiveness of those ads.
Through Facebook's experiment, people realized that they were using a product owned by a company whose incentives were vastly different from their own. And this discordance was uncomfortable. The experiment signaled the direction in which Facebook might permanently move -- a direction in which the already artificial experience of life via an EdgeRank-directed feed becomes more artificial as Facebook plays with that feed for the purpose of aligning your emotional state to its ad content.
On the flip side, whether OkCupid admits it or not, their incentives (and hence their experiments) are aligned with user goals. A dating site that advertises itself as one that maintains a subpar matching algorithm is a site that'll likely go out of business quickly. The company is incentivized ultimately to improve their matching algorithm, even if that means making it worse for a few users in the short term in order to run an experiment. Sure, some of OkCupid's experiments seem audacious, but the purpose was to better understand the factors that lead to successful matches on the dating website. And users get that.
I don't think most folks would have all websites cease testing their products on their users. That iterative improvement on user feedback is invaluable. Instead, what companies should consider is whether their tests are in line with user expectations and whether those tests serve to improve the user experience of the product. If there's a disconnect, then that should give a company pause.
This article is an experiment. Leave your comments in the comment section, and if you're not a bot trying to sell weight loss pills, who knows, maybe I'll change what I say based on your feedback.
RSS Feed