Growing Transparency in Research

Driven by a mutual dedication to transparency and accountability in research, Professor of Economics Richard Ball and Associate Librarian Norm Medeiros formed Project TIER in 2013. The initiative, whose name is an acronym for Teaching Integrity in Empirical Research, provides guidance to students conducting quantitative research to help ensure their work is transparent and reproducible. Earlier this summer, TIER held its fifth annual Fellows Conference, at which participants assessed the most effective means to expand the program’s reach.

“This was the most rich and fruitful discussion of all the years,” Ball said of this year’s conference.

The impetus for Project TIER came 19 years ago when Ball confronted a recurrent problem in his classroom: since students weren’t mandated to, they didn’t mentally keep track of the steps their empirical research went through, from raw data to the finished figures that appeared in their papers.

“It was often very hard to get students to explain to me what they’d done with their data, and that’s a problem,” Ball said. “If I asked them to start over with the original data and do it again, they’d get different figures.” 

To reach a solution, Ball approached Medeiros, the librarian for his subject area. After some brainstorming, the preliminary form of the TIER Protocol was born. The protocol details an alternate method of data management, wherein the students save their original dataset while recording each change made to it. The alterations are then coded into a script, which, when run on the original data, is able to reproduce the students’ final figures, while also allowing their instructors to easily see each step of the process.

“It made a big difference,” said Ball, reflecting on the protocol’s implementation in his classroom. “Not only did they understand data management, but they understood the statistics and the pictures and what it all meant.”

The primary concern of Project TIER has always been truly Haverfordian: a desire for true transparency between researchers and their audience. Since the initiative’s inception, that principle hasn’t been altered, but the focus has been on streamlining the protocol and ensuring that it can reach as wide an audience as possible.

“We want to build a community of instructors and librarians, who are really very aware of issues of reproducibility and are trying to promote better teaching of those practices,” said Medeiros.

To achieve this outreach, Project TIER, which is supported by the Alfred P. Sloan Foundation, has channeled its funding into the establishment of several annual fellowships awarded to academics across the country and beyond. Each year the ingoing and outgoing fellows convene for two days of presentation on their work and discussion on TIER’s mission. 

“We’re interested in finding out more systematically in colleges and universities across the U.S., what is typical about the way statistical methods are taught,” Ball stated.

Collaboration with one of the previous year’s fellows, Nicole Janz of the University of Nottingham has been instrumental in a transatlantic expansion of Project TIER’s mission to the U.K. TIER hopes to incorporate its transparency protocol into a preexisting national program “that’s meant to jumpstart quantitative methods training across U.K. universities at the undergraduate level.”

At this stage in its development, the effects of Project TIER are demonstrable, though not always visible. At academic conferences, both Medieros and Ball have stumbled across far-hailing strangers who have notified the pair of the TIER Protocol’s positive impact on their teaching.

“It’s been gradual, but satisfying all along,” said Ball, reflecting on their journey up to this point. “We never planned to do this. At first, it was just Norm and I with our PDF. We started out very homegrown, very small, and not sure anyone would care.”

With an unmistakably renovative approach to research, however, growth was inevitable. While the 2019 conference saw the pair focusing on a cultivation of that growth via their fellows, they believe that the best means of expansion is at its core pedagogical. 

“We're concerned about what's happening in the classroom and teaching good practices,” Medeiros said. “We hope that as these students go on to grad-school and become professors or go into the workforce that these habits that we’re teaching them, these ways of managing their work, will have positive effects downstream.”