Do you remember the Sokal affair? Mathematical physicist Alan Sokal submitted a fake paper titled “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity” to a postmodern cultural studies journal Social Text. Well, it was a real paper, but it was a hoax. It was written as a hoax, cobbling together the silliest tropes he could find about social construction of science and left-wing politics. Once it was published, he used it as a club to beat up on postmodern theory, and the humanities more generally. Many self-satisfied scientists claimed, at the time, that this confirmed all their worst suspicions about the emptiness of academic jargon-spinning in the humanities and social-scientists. The implication — sometimes made explicit — was that the reverse could never happen, that scientists know exactly what their terms mean, and could never be fooled by such a prank.
It turns out, the situation is much worse than that. Springer and IEEE have been forced to withdraw 120 papers that they published in various conference proceedings, and that turn out to have been randomly generated by the software SCIgen. Concerned to assure the public of the reliability of their peer-review process, Springer has now announced a technical solution: SciDetect, software that checks papers to determine whether they have been generated by SCIgen. (If it had been announced today I would have assumed that it was an April Fools prank, but the press release is from 23 March.)
Confirming me in my opinion that the whole system of peer review has outlived its usefulness, and is now living on as a vestigial parasite on the scientific enterprise.
Anyway, Springer has now thrown down the gauntlet, and young computer scientists should rise to the challenge of improving SCIgen, to fool the new software. We may see an accelerating arms race in scientific publishing, a kind of reverse Turing Test, with computers trying to fool other computers into believing that they are computer scientists. In the end, maybe the software will get so good that it will be doing original research and writing real scientific papers.
Common Infirmities 