Trends in the use of R & D output indicators in EC programme evaluation

Abstract

This paper rests upon a review of 15 evaluation reports of R & D programmes worked out during the 80's by the European Commission. The analysis aims at answering the main questions: Why did emerge the needs for output indicators in the middle of the 80's? What kind of output indicators were built up (or tentative)? With which methodology? What were their actual use in the evaluation reports? The linkage between EC R & D policies and evaluation is examined in order to discuss the relationships between the goals of R & D programmes and the criteria for evaluation. It is shown that the followed evaluation methodology and the evaluation goals at hand are paramount for the choice of output indicators; such goals encompass a.o. the description of the programmes, the assessment of the contractors opinion, the appraisal of the “techno-economic” effects of the programmes. As a result “expected output indicators” were developed (BRITE programme). On the other hand, one has called “meta-evaluation”, the indirect measurement of Scientific results by bibliometry (BEP_BAP programmes). Similarly, “intermediate indicators” were built up for evaluating the programmes management performance (ESPRIT programme). At last “derived output indicators” were used for techno-economic evaluation, (EURAM programmes) leading to the quantified global judgement of a “before-after” methodology, (SCIENCE-STIMULATION programmes).