The assistant professor of computer science and her collaborators will receive $35,000 to create a website that will help people identify and fix hidden biases in their data.
Haverford Assistant Professor of Computer Science Sorelle Friedler studies programming and prejudice. Last year she was part of a team that discovered a new way to see if computer algorithms—the kinds that determine what ads are suggested to website visitors or screen job applicants electronically—can be biased and a method for fixing them if they are.
The next step in her work is the creation of a website for journalists and members of the public that can help identify and fix hidden biases in their data. Such a site now is on its way to becoming a reality thanks to a $35,000 award from the Knight News Challenge on Data. Friedler and her collaborators are among 17 winners of the Challenge, chosen from 1065 applications, who were allocated a share of $3.2 million to develop projects that answer the question, How might we make data work for individuals and communities?
"Find out if your data discriminates," is Friedler's project's answer.
"I think there’s this idea that data is objective and somehow neutral and, therefore, obviously not discriminatory, and that’s not true," she says.
Although data gleaned from a computer algorithm doesn't have implicit biases, like people do, it can still use its code to reach conclusions, and those conclusions can be biased. And if that data is being used in a decision-making process, such as screening job applicants, that bias is causing discrimination.
"We're not assigning motive or malicious intent to this algorithm, but we are saying if you use this algorithm, you might be discriminatory, and you should know that, and you should do something about it," says Friedler. "It's not because you’re a bad person, it’s because the algorithm has been trained to exploit any pattern that it finds, and so if you want it not to exploit a pattern that is discriminatory, you have to go out of your way to make sure that that happens."
One such way Friedler and her collaborators—the University of Utah's Suresh Venkatasubramanian and Data & Society Research Institute colleagues Wilneida Negron and Surya Mattu—imagine this new website being used is by journalists to research stories such as Mattu's recent ProPublica expose on Princeton Review's biased pricing. The report uncovered an unexpected side effect of the company's geographic approach to pricing for its online SAT tutoring packages: Asians are twice as likely to be offered a higher price than non-Asians.
"Surya had to run a bunch of logistic regression analysis to try to figure that out," says Friedler of the technical research methodologies that led to the story. "And using our work we were able to show the same thing he had already figured out using a bunch of other tools… That confirmed for us that [our] method would be usable to write that type of story and would make it possible for someone without Surya’s really extensive technical knowledge to do that type of analysis."
Friedler and her collaborators will use their Knight News Challenge award to fund the work Haverford and University of Utah students, who will build this new website, and to eventually pay for some extensive user testing once it's finished. Derek Roth '17 is starting to program the site this semester and then Friedler will hire another student to continue the work this summer. The team aims to have a finished site by July.
