Robert Ambrogi Blog
Cite Wikipedia in an argument to a judge, and you are likely to get a disapproving glare in response. “Give me solid precedent,” the judge is likely to say, “cases and statutes and respected authorities.”
But a team of scientists from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and Maynooth University, Ireland, has concluded that, in fact, Wikipedia has a measurable impact on judicial decision making.
They describe their findings in a new paper released today, Trial by Internet: A Randomized Field Experiment on Wikipedia’s Influence on Judges’ Legal Reasoning.
Influence on Decisions
To assess whether Wikipedia impacts judicial decisions, the researchers set out to test for two types of influence: (1) whether the creation of a Wikipedia article on a case leads to that case being cited more often in judicial decisions; and (2) whether the text of judicial decisions is influenced by the text of the corresponding Wikipedia article.
To conduct this assessment, the researchers actually created new Wikipedia articles to examine how they affected the legal decisions of judges in Ireland. Specifically, they enrolled law students to write some 150 Wikipedia articles on Irish Supreme Court decisions. Half the articles were randomly chosen to be uploaded to Wikipedia, where they could be accessed by judges, clerks, lawyers, and others.
The other half were kept offline, and this second group of cases, the researchers said, provided the counterfactual basis of what would happen to a case absent a Wikipedia article about it.
What they found is that the addition to Wikipedia of an article about a case increased a case’s citations by more than 20%. The increase was particularly strong for cases that supported the argument the citing judge was making in the decision. The increase was larger for citations by lower courts, and mostly absent for citations by appellate courts.
A Crutch for Busy Judges?
This shows, the researchers suspect, that Wikipedia is used more by judges or clerks who have a heavier caseload, for whom the convenience of Wikipedia is more appealing.
The study also used natural language processing to perform a semantic comparison of judicial decisions to Wikipedia articles about cases cited in those decisions. The researchers found that a textual similarity exists between the judicial decisions and the Wikipedia articles.
This reveals, they say, “that the contextualization of the case by law students on Wikipedia is itself influencing judicial reasoning.”
“A case’s inclusion on Wikipedia correlates with its judicial citation in a way that suggests a direct influence,” the paper concludes. “Similarly, we find that the text of judicial opinions correlates with that of Wikipedia case entries, and that this correlation extends beyond passages quoted from the pertinent case to encompass Wikipedia’s legal analysis itself. Taken together, these results indicate that, sometimes, judges are deferring to Wikipedia rather than applying their own legal expertise as they craft decisions.”
MIT researcher Neil C. Thompson, the lead author of the paper, said that, to his knowledge, this was the first randomized field experiment that investigated the influence of legal sources on judicial behavior.
“And because randomized experiments are the gold standard for this type of research, we know the effect we are seeing is causation, not just correlation,” he said. “The fact that we wrote up all these cases, but the only ones that ended up on Wikipedia were those that won the proverbial ‘coin flip,’ allows us to show that Wikipedia is influencing both what judges cite and how they write up their decisions.”
Public Policy Issue
Thompson said that the results also highlight an important public policy issue. If a source is as widely used as Wikipedia, it is important to find ways to ensure that the information is of the highest quality.
“The finding that judges or their staffs are using Wikipedia is a much bigger worry if the information they find there isn’t reliable,” he said.
Although the study focused on Ireland, it concludes that the impact of Wikipedia on judicial decision-making in larger common-law jurisdictions such as the United States may be even greater.
“For the moment this effect in Ireland is limited, because the number of Irish court judgments on Wikipedia remains small,” the paper says. “But in other jurisdictions, such as the United States, many more cases have already been added to Wikipedia.
“This suggests that, across many common-law jurisdictions, Wikipedia (and, by extension, the legal analysis of unknown internet users) might already be playing an important role in shaping judicial decisions.”
A Gap that Needs to be Filled
The bottom line, the authors say, is that this research highlights a gap that needs to be filled.
“Judges need an easily accessible source of knowledge that is also authoritative,” the paper concludes. “Policy-wise, this could be addressed by buttressing the reliability and review of Wikipedia content by including legal professionals as supervising editors to certify page quality, or by augmenting the content of authoritative but less-broad sources, and using those for the provision of legal information about particular jurisdictions.”
As a possible model for this in the U.S., the authors reference the Oyez Project, which, in collaboration with Justia, offers free synopses of recently published decisions of the U.S. Supreme Court and of the courts of appeal that, unlike those of Wikipedia, are written and editorially supervised by legal professionals.
“Our experiment reveals that initiatives along these lines would be valuable and that they might help protect the foundational expectation of how legal decisions ought to be made: that they should be based on carefully-considered expertise about what laws and precedents say, and never be at the mercy of internet ghost-writers.”
The paper was written by Thompson, together with Maynooth University researchers Brian Flanagan, Edana Richardson and Brian McKenzie, and Xueyun Luo of Cornell University. It will be published as a chapter in the forthcoming book, The Cambridge Handbook of Experimental Jurisprudence, from Cambridge University Press.