Thursday, April 21, 2005
academic susceptibility to information fraud; list of other blog entries on information / communication / knowledge
MIT prank paper accepted for publication
Thursday, April 21, 2005 Posted: 9:40 AM EDT (1340 GMT)
BOSTON, Massachusetts (AP) -- Three MIT graduate students set out to show what kind of gobbledygook can pass muster at an academic conference these days, writing a computer program that generates fake, nonsensical papers. And sure enough, a Florida conference took the bait. [more...]
good stuff. but the focus seems only to be on the inadequacy of the conferences, etc. however, it also implicitly points up the problem with the producers of papers as well: so much of it is indistinguishable from crap, because of multiple factors, including lack of content quality or originality, content obscurity, poor style or writing, and huge overall volume of information (reviewers being swamped in a reverse-snr effect, unable to pick out the outliers, in addition to the usual swamping of high-value papers).
i've seen this sort of thing before, various generators, i mean. it hadn't occurred to me to attempt to submit anything from them. come to think of it, such procedures should be instituted for all journals, conferences and any other forum where reliability of content is an issue. it would provide a type of control as to the robustness of adjudication processes. such tests should include not only the generated stuff, but cases covering all possible combinations of errors in reviewing. it's sort of like having people testing the security at airports.
hmm - since many preprint archives are completely unreviewed, it should be possible to flood them with pseudo-science spam. yuk, but perhaps a necessary step.
in fact, it may turn out that much of what is churned out in today's publish-or-perish world of academia would actually fail such tests. through this forced artificial criteria for tenure competition and what have you, low quality/content work is encouraged over more valuable criteria. perhaps using automated tools along these lines would negate these practices and loosen up the intellectual gridlock now in place.
there have recently been tools in the news that have traced spam and the evolution in content of specific instances (chain letters in particular), and also teachers using automated tools to detect plagiarism. adding the above mechanisms, and other concepts, to this toolbox can only help.
but oh, how the university system would scream. change is never easy for anyone.
thought: what about such testing for our court systems as well ?
related posts - i'll take this opportunity to bring together references on information etc. scattered throughout this weblog:
- danger, will robinson ! - information reliability
- the truth, the whole truth, and nothing but the truth - up to a limit - the effects of privacy
- [this title is not important] - truth ?
- as pogo said... - speech as a right vs. privilege
- i know that you know that i know that ... - limits to knowledge
- don't you love a good conspiracy theory ? - on propaganda
- q & a - the unreasonable persistence of state (see comments)
- solaris - communications on the edge
- alt mbti ? - analyzing personality and pseudo-math
- discernment - misinformation and manipulation
- wikinews ? - modes of communication
- cross link - censorship
- using global intelligence to provide global intelligence - inputs from the webmind
- some thinking about thinking - religion as viral meme
- fuck you - more censorship
- brief visit with an old passion - perspectives
- thoughts on thought - conformity in science
- quick, look over here - again perception, and conformity
- meta-musings: on communication - shifting channels
- not a new idea... - perceptions (again)
- run that by me again ? - a ramble towards reality
- the state of scifi - perception (again)
- the age of metaknowledge - abstraction
- feng shui for the mind - the state of education
- the "information poor" get a boost - access
clearly this needs another category. what should it be called ? and i noticed a space related category for a number of other posts as well. maybe now that i'm at the 300 post threshhold i should look at categorizations again. to actually apply them i wshould use another interface altogether. probably ms access ;-)
update - ok, here's the categories:
- i.c.k. - “information, communication and knowledge”
- e.t. - “(extra)terrestrial”
that last one still needs to be appplied to previous posts. see the category pages for a little more explanation. sorting all this crap out is a long term effort, but this should capture a good chunk of the material so far - once i go back and apply them, anyway. there's also room for a category for things with a vague resemblance to “humor”, and another that is “dear diary” stuff. and ... ?