Quattrone Center’s inaugural summer internship program gives law students opportunity to conduct criminal justice empirical research

Since its inception, the Quattrone Center for the Fair Administration of Justice has strived to use interdisciplinary, data-driven research to tackle some of the most pressing problems in the criminal justice system.

After Minneapolis police killed George Floyd in May, sparking nationwide demands for police accountability and justice, project requests began to pour into the Quattrone Center. Associate Dean and Executive Director John Hollway said that he understood the increase in requests to be a testament to both the trust people had in the Center’s capabilities and the widespread desire for positive change.

“A lot of jurisdictions turned to us and said, ‘help us respond to our community’s calls to reform,’” Hollway said. “So the law students got really substantive opportunities.”

University of Pennsylvania Carey Law interns worked on three Quattrone Center projects over the summer: a sentinel event review with the Tuscon Police Department; a comparison of the efficacy of public defender office models; and a data analysis of prosecutorial misconduct in Pennsylvania. Here we take a closer look at each project and the roles law students played in bringing them to fruition.

Tucson Police Department sentinel event review

Over the summer, Adam Markovich L’22 and Cassidy Rowe L’22 worked closely with the Tucson Police Department, which had reached out to the Quattrone Center after two separate deaths of individuals in police custody. The law students were tasked with helping develop a “sentinel event review,” which Hollway describes as, “[an] evaluation of a case with undesired outcomes where we look back at the root causes and contributing factors.”

Hollway further explained the sentinel event review concept as it applies to criminal justice.

“Obviously, police officers obviously don’t want people to die in custody,” said Hollway. “The question is why does that happen if nobody wants that to happen? What are the factors that came together to cause this outcome that nobody wanted?”

Markovich and Rowe worked closely with the Tucson Police Department to gather research used in the sentinel event review. After signing non-disclosure agreements, Markovich and Rowe were granted access to privileged materials such as case files, body camera footage, transcripts, and 911 call recordings.

“The Tucson Police Department was so forthcoming,” Rowe said. “Essentially what they’re doing is opening themselves up to a lot of criticism in a time when society’s view of police is very critical. It was an important step in accountability and transparency for them to give us access to all of their stuff.”

In addition to analyzing the police department’s internal records, Markovich and Rowe also researched law enforcement strategies employed across the country in an effort to figure out which practices yielded the best results.

Using Markovich and Rowe’s research, the Quattrone Center convened a Tucson community stakeholder board that included police officers, firefighters, 911 dispatchers, community activists, representatives from the police union, and a critical care nurse. Over the course of the summer, Markovich and Rowe coordinated and moderated four separate eight-hour meetings during which stakeholders discussed the multi-facetted issue of death in police custody and the law students presented their research, shared their observations, and listened to the community’s feedback.

“Each week, we met with the whole board to discuss where we were with the research. We talked about what they found to be important and what we found to be important,” Markovich said. “So a lot of their voices, especially those of the community leaders, are worked into the report.”

Markovich and Rowe have continued to work with the Quattrone Center during the school year. Though they plan to take on more projects as the year progresses, their first task was to finalize the report for the . The report includes the group’s conclusions as to how the Tucson Police Department and other parts of the criminal justice system could improve to prevent future tragic instances of death in police custody.

“The idea of the sentinel event reviews is not to lay blame, but to have a blend of forward and backward-looking accountability,” Rowe explained. “You’re identifying contributing factors to these bad outcomes. And by not assigning blames, you get a lot more buy-in from the participants. You’re not condemning them. You’re coming together and saying this is a systems failure. This is not an individually-blamed issue.”

To Markovich, part of what makes the Quattrone Center unique is its systems-based approach to problem solving.

“Making proper change is hard. It requires looking at the overall system and making more nuanced or difficult changes,” Markovich said. “That’s the goal of our report: to change the processes these officers are operating under to minimize the chance that something like this happens again.”

Overall, the experience was incredibly positive for the law students.

“We were constantly thrilled with what we were doing,” Markovich said. “Our work was important. It felt good. It was engaging and interesting. Everyone was there to support us to keep working on it.”

Comparison of the efficacy of public defender office models

Quattrone Center Senior Fellow and Academic Director Paul Heaton wondered whether the organizational design of public defender offices could be leveraged to yield more favorable results for clients. For the past several years, he has been working to answer this question with Pamela Metzger, a law professor at Southern Methodist University.

Initially, Heaton and Metzger hypothesized that discrepancies in trial outcomes might be correlated to whether a public defender office is organized “vertically” or “horizontally.” In a “vertical” public defender office, the same public defender litigates a case throughout the entirety of its duration and appeals. By contrast, in “horizontal” public defender offices, the case is transferred as it moves through the system to attorneys that specialize in specific stages of litigation.

Testing this hypothesis posed a challenge. Generally, jurisdictions only have one public defender office. To compare one public defender office to another, differently-structured public defender office in a second jurisdiction generates too many variables to draw clear conclusions.

“It’s something that is really hard to study, because you can’t generally do an apples to apples comparison in the real world,” Hollway said.

Fortunately, Heaton, Metzger, and this summer’s interns found their solution in King County, Washington. Until recently, public defense in King County operated out of four law firms. The firms merged into one office a few years ago but still operate separately. Prior to the merge, two of the offices had been operating in a vertical structure, and two had been operating in a horizontal structure. It was because of this unique situation that researchers at the Quattrone Center were afforded the rare opportunity of studying the two systems in parallel.

As they continued to gather and analyze information, however, the team realized that their ideal research scenario was actually more complicated than they may have anticipated.

“The firms actually weren’t so clearly structured,” said Eleanor Allen L’22, who worked on the project with Heaton and Metzger. “None are purely horizontal and none are purely vertical. In general, they’re kind of all hybrid.”

Despite the conclusion that all four firms existed in some gray area between “vertical” and “horizontal,” the outcomes the firms achieved for their clients, such as conviction rates and sentencing durations, varied greatly.

The researchers switched gears and began to question if other structural elements might yield discrepant outcomes.

This summer, Allen devoted much of her time to investigating whether a firm’s values and culture have an impact on its success rate. To illustrate what the researchers meant when they discussed firm values and culture, Allen explained that one firm might consider itself as “really strong with client communication” and pride itself on “always being in touch with their clients and really involving them in the process.” Another firm might prioritize “being tough negotiators” and “not settling.” The researchers’ task was to compile this information and examine it through a mixture of both qualitative and quantitative data analysis.

The qualitative research-gathering pushed Allen, who has a background in economics, out of her comfort zone. A large part of her role this summer involved interviewing King County Superior Court judges about their perceptions of the public defense system. She also spent time partnering with another researcher, Andy Davies of SMU, to design a survey for the King County public defenders. The researchers hope that these methods might yield additional information to explain the observed differences between the public defense divisions.

Once the survey results are in, the research team plans publish their findings in an academic journal next year.

Reflecting on the project, Allen noted that the research would never have succeeded without the flexibility of the Quattrone Center’s research team.

“I think the pivot in the main question that we were asking is a really great reminder for law – and life – that sometimes you set out on something that seems clear cut but actually ends up being more nuanced that you thought it was,” she said. “In theory it would be really great to compare horizontal and vertical systems of public defense, but once you start to really dig in, you see: Is there such a thing as a solely vertical organization? Seeing that these things are a little bit more gray than they are on paper is a really great takeaway.”

Data analysis of Pennsylvania prosecutorial misconduct

In addition to projects in partnership with external clients, the Quattrone Center also devises independent academic research. This summer, Assistant Director Ross Miller led the Quattrone Center’s ongoing efforts to analyze prosecutorial misconduct allegations in Pennsylvania. To do this, Miller worked with interns to read and code 16 years’ worth of relevant Pennsylvania appellate records. Their two-part goal was to (1) identify different categories of prosecutorial misconduct allegations and (2) create empirical descriptions that could be used in future research and policy development.

Miller explained that, though the media often uses the term “prosecutorial misconduct,” many people do not fully understand what the term means or what it might look like in practice.

“It tends to be something that’s very anecdotal. You have an exoneration and some really bad conduct by a particular individual. But that doesn’t really tell you that much about how those cases move through the system,” Miller said. “The purpose of the project was to analyze and describe what the world of those claims looks like in the context of Pennsylvania, and then provide some further information about what the systems are outside of Pennsylvania that are designed to deal with that problem, to recognize it and to remediate it.”

Penn Law students had already been assisting the Quattrone Center in compiling a coded prosecutorial misconduct database. This summer, the interns pushed the work further by supplementing the database with additional research from external sources, such as the National Registry of Exonerations, which helped to contextualize what was going on in Pennsylvania with what had been reported in other states. Using this data, the interns were then tasked with producing narratives that explain the data in a way that is digestible for both legal and non-legal readers.

Kristen Ierardi L’22 enjoyed that this project allowed her to use the writing skills she developed as a history major. She felt that creating narratives helped transform the data and caselaw into more compelling, interesting pieces of advocacy.

“Legal writing is what it is for a reason. You need to have things fit within a narrow, procedural structure,” she said. “It was nice to be able to read these cases, synthesize them, look at the data, and write a story with what I thought were the most important and telling pieces.”

The Quattrone Center anticipates that the report from this project will be useful not only to prosecutors but also to a variety of stakeholders throughout the criminal justice system. Judges may use the data to better understand how certain prosecutorial misconduct claims exist in larger contexts, which could help them write more specific opinions. Practitioners may use the data to situate a client’s claim alongside others that have been similar, which could allow them to better understand the law surrounding that particular situation. Beyond that, the Quattrone Center also hopes that organizations, such as the Disciplinary Board of Pennsylvania, can use the report to advocate for legislative reform of policies related to prosecutorial misconduct.

When doing the research and writing the report, Ierardi said that it was “frustrating” to watch misconduct unfold and go unnoticed – sometimes for years at a time and through several rounds of appeals. This year, Ierardi plans to channel that frustration into pro bono work with the Criminal Records Expungement Project.

“It’s a very unfair system,” she said, reflecting on the multitude of stories and cases that she read this summer. “To be able to help someone to get a second chance or a clean slate is super important.”

Quattrone Center summer intern experience

To Miller, the most rewarding aspect of the summer program was getting to see students engage with contemporary criminal justice issues on a granular level. Throughout the summer program, this engagement happened both through the work the interns completed and through informal social events.

Despite the necessary restrictions of running a summer program during a pandemic, the Quattrone Center still managed to arrange widely popular weekly “brown bags,” which turned into a hallmark aspect of the summer for many of the interns and staff members. “Brown bags” were informal lunchtime Zoom meetings meant to provide a space to build relationships, bounce ideas around, and process the momentous events related to police and justice reform that continued to unfold across the country throughout the summer.

“Sometimes they were light conversations about your favorite 90s snack food,” Ierardi said about the brown bags. “But other weeks, they dove into some other important, tough subjects.”

Rowe echoed this sentiment, laughing as she recounted the “random things” the group chit-chatted about some lunches; at one particularly memorable brown bag, Miller discussed his experience at a prior job designing platypus food. Yet, on other occasions, the conversations turned serious.

“We’d also talk about social justice reform,” Rowe said. “It was definitely difficult. There’s that start-stop on Zoom, and it’s harder to read people. But they somehow created a space still where we could have those conversations.”

Hollway was impressed by the interns’ enthusiasm in talking about racism and white privilege, especially in light of the national conversation about how those issues play into the justice system.

“Summer started intensely,” Hollway said. “We had some pretty impactful and soul-searching conversations about racial inequities and how to respond to them and about white privilege. Over the summer, we watched [the interns] become more comfortable engaging in that conversation and really become part of what the Center does. Part of what we do in an academic space is create an environment where people can have those conversations – and have it be a psychologically safe environment.”

Rowe described the Quattrone Center as “a very forthcoming and honest space,” crediting its leadership with creating an atmosphere in which challenging conversation topics were both encouraged and accessible.

Markovich agreed, adding that, beyond the brown bags, he was impressed with the leadership’s connectivity even when the office functioned remotely due to COVID-19.

“I was never hesitant or felt that it was difficult to reach out to anyone in the Quattrone Center with a question or for help,” Markovich said. “It was never an issue. I could always pop in and chat.”

Ierardi echoed this in describing the internship as being both a “personal” and “collaborative” experience, in spite of the fact that everyone was physically distant.

Overwhelmingly, the interns credited the Quattrone Center’s leadership with facilitating an enriching and rewarding internship experience.

“I cannot give the Center enough credit for what a wonderful experience everyone made this because they absolutely gave us some very important things and basically said, ‘Run with it. Let us see what you can do,’” said Rowe. “It never felt like we were doing busy work or just supplementing the things they were doing. We got to work directly with police departments, and that opened my eyes to how you can make bigger impacts as an individual.”

Read more about the Quattrone Center and its interdisciplinary, data-driven approach to solving problems in the criminal justice system.