In article <371A29F2.2E1C at post.queensu.ca> forsdyke at post.queensu.ca writes:
>Certainly a practicing physician is not going to have
>time to read the primary literature.
Without peer review there would be no primary literature to base
a secondary literature on. The refereed journal literature is the
primary research literature.
>Cancer researchers today pick what
>is perceived as the latest break-though because the journals have
>accredited it. Cancer researchers TOMORROW will sift through the
>electronic literature using a multiplicity of search tools, including
>authors' track-records,
If track records were a sufficient basis for selection, referees
would have a much easier time of it; so would editors.
>and (if the author has sought it), the
>approbation scores contributed by peer-reviewers picked by the
>appropriate scientific society.
Are you imagining that everything anyone elects to post on the
Net is eligible for "scientific society approbation"? Referees
will now be at the beck and call of anyone who knocks?
Referees are already a scarce and over-used resource in classical
peer review (and that is when it is a known and competent editor
or grant officer who importunes them). You are imagining them
lending their services just to give "scores" to anyone, with not
even publication being contingent on being answerable. This kind
of proposal has been much debated lately in:
<http://amsci-forum.amsci.org/archives/september-forum.html>
>If cancer researchers judge the work
>important they will set out to confirm the results, and their
>confirmatory papers will, in turn, appear in the electronic literature.
This assumes that they have the time and inclination to sort through
the unprocessed free-for-all for a candidate in the first place, and
no better way to judge what to invest not only their reading and
refereeing time into, but now also their experimental research time.
>These will be read by those writing reviews of promising new therapies,
>usually people with high reputations in the field, and so the work will
>come to the attention of practicing physicians.
Currently the people with high reputations writing reviews are basing
their reviews on the refereed primary literature; you are assuming
(unrealistically) that they could or would do so on the basis of
raw, unfiltered postings (guided only by author track record?), and,
even if this were feasible, that this would result in a secondary
literature of the same reliability as the one we have now, grounded
firmly and fully in a refereed primary literature. (Why does this
call to mind the Cheshire Cat's smile, hanging from a skyhook?)
>> > Actually, I find journal peer-review as currently practiced more akin
>> > to censorship than quality control.
>>>> It would be interesting to hear how many years of experience in editing
>> that finding was based on: How much wading through raw manuscripts to
>> sort out what is worth reading and trying to build upon has led you to
>> this conclusion, and whatever your trick is, please share it with us!
>>Well, over three decades of interacting with editors and reviewers (both
>as author and reviewer) has given me some feel for the matter.
You have not even faintly felt the problem until you have tried to
process unrefereed, submitted papers; and even that is a more reliable
literature than we would face if the invisible hand of peer review were
not in the background as it is for submitting authors now. Without
answerability, overall quality can only go in one direction.
>The peer
>reviewing of papers (usually without reimbursement) takes up much of the
>time of busy people, who thus have less time for other matters
>(preparing lectures, reading the literature, treating patients,
>research, even fishing).
But these busy people have more time to make their way instead
through unfiltered sludge? Are those who have the time for that
even the ones we want to trust to filter the literature for us?
>Much of the effort, both on the part of
>reviewers and of the author responding to reviewers, is, in my
>experience, of little value.
Unfortunately it is not individual authors' experience that is
in a position to make this judgment; it can't even be made from
the referees' standpoint. The only one in a position to judge
what difference refereeing makes is the editor who processes
the raw submissions -- except if peer review really is abandoned
in favour of the type of system you propose, for then we would
all get a taste of what an unfiltered corpus would be like, without
the mediation of referees and editor; and not even that, for with
the "invisible hand" gone, even the unfiltered baseline would start
to fall, no longer having to anticipate answerability (except to
journeyman reviewers with nothing better to do with their time).
>If I need help in writing a paper, I
>consult colleagues both within my institution and elsewhere, BEFORE I
>even consider submitting. Subsequent peer review adds little to this.
Informal peer advice before submission is pretty much the norm already,
yet no one has mistaken it as a substitute for peer review; on the contrary,
it is presupposed, and the papers that have not even been looked at
by peers before submission usually fare much worse with referees, for
good reasons. We all need feedback; but we are not answerable to our
friends' and colleagues informal suggestions as we are to a journal
editor in formal peer review. And that anwerability is the critical
difference between the informal and the formal phases of the
entire self-corrective process.
>> Human nature being what it is, it takes
>> the path of least effort when it can, regressing on the mean or even
>> meaner.
>> So what? Let them publish in the electronic media. There is plenty
>of room. Their work will be disregarded, but who cares? Why should they
>be "answerable"?
Let them archive publicly in the elctronic media, by all means.
But let's reserve the word "publication" for a literature that
has been certified to have met a standard with which we are familiar.
Otherwise every posting is on a par, and a pig in a poke (even given
what we might be able to infer from "track records").
> I do not personally have the time to troll through all those
>> least-effort products and find what (if anything) still meets the
>> standards (such as they are) of the refereed, tagged literature of
>> today.
>> Then don't! Await the secondary reviews by accredited experts in the
>field.
Today's secondary reviews are done by accredited experts on a refereed
primary literature. Who would be doing what in the free-for-all you
envision, no one can say, but, prima facie, I would be almost as
worried about trusting these new "experts," with all this trawling
time and inclination, as I would be about about having to trawl it myself...
--------------------------------------------------------------------
Stevan Harnad harnad at cogsci.soton.ac.uk
Professor of Cognitive Science harnad at princeton.edu
Department of Electronics and phone: +44 1703 592-582
Computer Science fax: +44 1703 592-865
University of Southampton http://www.cogsci.soton.ac.uk/~harnad/
Highfield, Southampton http://www.princeton.edu/~harnad/
SO17 1BJ UNITED KINGDOM ftp://ftp.princeton.edu/pub/harnad/
