We aren’t saying that every innovation requires A/B testing. Nor are we advocating nonconsensual experiments involving significant risk. But as long as we permit those in power to make unilateral choices that affect us, we shouldn’t thwart low-risk efforts, like those of Facebook and OkCupid, to rigorously determine the effects of those choices. Instead, we should…applaud them.
Meyer offers more perspectives on the issue in her interview with Nicolas Terry and me on The Week in Health Law podcast.
For an alternative view, check out my take on “Facebook’s Model Users:”
[T]he corporate “science” of manipulation is a far cry from academic science’s ethics of openness and reproducibility. That’s already led to some embarrassments in the crossover from corporate to academic modeling (such as Google’s flu trends failures). Researchers within Facebook worried about multiple experiments being performed at once on individual users, which might compromise the results of any one study. Standardized review could have prevented that. But, true to the Silicon Valley ethic of “move fast and break things,” speed was paramount: “There’s no review process. Anyone…could run a test…trying to alter peoples’ behavior,” said one former Facebook data scientist.
I just hope that, as A/B testing becomes more ubiquitous, we are well aware of the power imbalances it both reflects and reinforces. Given already well-documented resistance to an “experiment” on Montana politics, it’s clear that the power of big data firms to manipulate even the very political order that ostensibly regulates them, may well be on the horizon.