Limitations of citation analysis

An observation at the CWTS Graduate Course Measuring Science: in most lectures, the presenters emphasize not only how indicators can be constructed, measured, and used, but also under what circumstances they should not be applied. Thed van Leeuwen, for example, showed on the basis of the coverage data of the Web of Science that citation analysis should not be applied in many fields in the humanities and social sciences, and certainly not for evaluation purposes. If the references in scientific articles in the Web of Science are analyzed, there are strong field differences in the extent to which they cite articles that are themselves covered by the Web of Science. In biochemistry this is very high (92 %), whereas in the humanities this drops to below 17 %. Since citation analysis is almost always based on Web of Science data, most relevant data on communication in the humanities is missed by citation analysis. Of course, this is well-known and it is the usual argument in the humanities and social sciences against the application of citation analysis. However, this also has meant that most scholars see CWTS principally as associated with any use of citation analysis. CWTS does currently not have a strong reputation as the source of critique of citation analysis, although it has systematically, at least since 1995, criticized the Impact Factor and has also been very critical of the very popular and equally problematic h-index. Interesting mismatch between practice and reputation?

Teaching scientometrics and bibliometrics

Yesterday, the annual Graduate Course Measuring Science started here at CWTS. 24 PhD students and professionals from the information industry (publishers and software houses) are taking week-long a crash course in bibliometrics and scientometrics. Virtually all researchers at CWTS are teaching one or more slots, which gives the students the unique opportunity to get a firm grip on the field from a variety of angles and perspectives. For me, this is a convenient way of immersing myself in the way scientometrics is being done at CWTS and to look at the various methodological debates in the field from the perspective of CWTS. First impression yesterday: the students were bombarded with quite a lot of data and empirical findings, which they seemed to take up calmly. No furious debates yet. But it was only the opening day, so who knows? I am going to discuss the work on modelling the peer review system today, let us see how this goes.

Scientometric chicken

The beautiful building where CWTS is housed is part of Leiden’s cultural heritage. It was the first physiological laboratory of the university, hence it is named after Willem Einthoven. In this area a couple of proud cocks roam the area, quite beautiful and confident animals.





 Until recently, they had not seemed to respond storngly to people. However, the day before yesterday, as I approached, they immediately ran over and clearly expected me to feed them. It was the day of the opening of the academic year, so I guessed someone in a black suit (which I was wearing for the occasion) must have the habit of feeding them. As I left them, they seemed genuinely disappointed.