A recent story in the New York Times provides a contemporary snapshot of how Internet-based recognition metrics are challenging the closed peer review typical of traditional academia.
“Knowledge is not democratic,” said Michèle Lamont, a Harvard sociologist who analyzes peer review in her 2009 book, “How Professors Think: Inside the Curious World of Academic Judgment.” Evaluating originality and intellectual significance, she said, can be done only by those who are expert in a field.
At the same time she noted that the Web is already having an incalculable effect on academia, especially among younger professors. In her own discipline, for instance, the debates happening on the site Sociologica.mulino.it “are defined as being frontier knowledge even though they are not peer reviewed.”
The most daunting obstacle to opening up the process is that peer-review publishing is the path to a job and tenure, and no would-be professor wants to be the academic canary in the coal mine.
The first question that Alan Galey, a junior faculty member at the University of Toronto, asked when deciding to participate in The Shakespeare Quarterly’s experiment was whether his essay would ultimately count toward tenure. “I went straight to the dean with it,” Mr. Galey said. (It would.)
Although initially cautious, Mr. Galey said he is now “entirely won over by the open peer review model.” The comments were more extensive and more insightful, he said, than he otherwise would have received on his essay, which discusses Shakespeare in the context of information theory.
Advocates of more open reviewing, like Mr. Cohen at George Mason argue that other important scholarly values besides quality control — for example, generating discussion, improving works in progress and sharing information rapidly — are given short shrift under the current system.
“There is an ethical imperative to share information,” said Mr. Cohen, who regularly posts his work online, where he said thousands read it. Engaging people in different disciplines and from outside academia has made his scholarship better, he said….
“We have never done it that way before,” should be academia’s motto, said Kathleen Fitzpatrick, a professor of media studies at Pomona College.
This thawing out of academia’s hoary conventions for evaluating scholarship is good news to researchers drawn to more networked approaches such as CommentPress or ThoughtMesh.
For its part, the New York Times warns that that “many professors, of course, are wary of turning peer review into an ‘American Idol’-like competition.” But one startup, PeerIndex, seems to have taken that challenge as a rallying cry:
Celebrities like Britney Spears may be the ‘most followed’ on Twitter, but new service PeerIndex mines the content of tweets and tracks the spread of links and phrases to reveal the hidden experts in specific areas, from cloud computing to venture capital. The authorities the site finds for a given subject often have only a few hundred followers, but the content of their tweets is known to spread widely.