---------- Forwarded message ----------
Date: Sun, 24 Nov 2002 20:10:34 -0500
From: Peter Suber <peters at earlham.edu>
To: September 1998 American Scientist Forum
<SEPTEMBER98-FORUM at LISTSERVER.SIGMAXI.ORG>
Subject: Re: UK Research Assessment Exercise (RAE) review
In the recent postings on RAE ratings and scientometrics, I don't believe
I've seen anyone cite this piece of research:
Andy Smith and Mike Eysenck, "The correlation between RAE ratings and
citation counts in psychology" (June 2002)
http://psyserver.pc.rhbnc.ac.uk/citations.pdf
The authors' summary: We counted the citations received in one year (1998)
by each staff member in each of 38 university psychology departments in the
United Kingdom. We then averaged these counts across individuals within
each department and correlated the averages with the Research Assessment
Exercise (RAE) grades awarded to the same departments in 1996 and 2001. The
correlations were extremely high (up to +0.91). This suggests that whatever
the merits and demerits of the RAE process and citation counting as methods
of evaluating research quality, the two approaches measure broadly the same
thing. Since citation counting is both more costeffective and more
transparent than the present system and gives similar results, there is a
prima facie case for incorporating citation counts into the process, either
alone or in conjunction with other measures. Some of the limitations of
citation counting are discussed and some methods for minimising these are
proposed. Many of the factors that dictate caution in judging individuals
by their citations tend to average out when whole departments are compared.
Peter
----------
Peter Suber, Professor of Philosophy
Earlham College, Richmond, Indiana, 47374
Email peters at earlham.edu
Web http://www.earlham.edu/~peters
Editor, Free Online Scholarship Newsletter
http://www.earlham.edu/~peters/fos/
Editor, FOS News blog
http://www.earlham.edu/~peters/fos/fosblog.html
