|
Research performance evaluation: the experience of an independent medical research institute
Catherine C.
Schapper A,
Terence
Dwyer A D,
Geoffrey W.
Tregear B,
MaryAnne
Aitken A and
Moira A.
Clay C
A
Murdoch Childrens Research Institute, Royal Children’s Hospital, Parkville, VIC 3052, Australia. Email: cathy.schapper@ranzcp.org; maryanne.aitken@mcri.edu.au
B
Florey Neuroscience Institutes, Howard Florey Institute, Level 2, Alan Gilbert Building, 161 Barry St, Carlton South, VIC 3053, Australia. Email: geoffrey.tregear@florey.edu.au
C
Telethon Institute for Child Health Research, PO Box 855, West Perth, WA 6872, Australia. Email: mclay@ichr.uwa.edu.au
D
Corresponding author. Email: terry.dwyer@mcri.edu.au
Australian Health Review
36(2)
218-223 http://dx.doi.org/10.1071/AH11057
Submitted: 10 June 2011 Accepted: 22 September 2011 Published:
25
May
2012
|  |
|
|
Abstract
Background.
Evaluation of the social and economic outcomes of health research funding is an area of intense interest and debate. Typically, approaches have sought to assess the impact of research funding by medical charities or regional government bodies. Independent research institutes have a similar need for accountability in investment decisions but have different objectives and funding, thus the existing approaches are not appropriate. Methods.
An evaluation methodology using eight indicators was developed to assess research performance across three broad categories: knowledge creation; inputs to research; and commercial, clinical and public health outcomes. The evaluation approach was designed to provide a balanced assessment across laboratory, clinical and public health research. Results and discussion.
With a diverse research agenda supported by a large number of researchers, the Research Performance Evaluation process at the Murdoch Childrens Research Institute has, by necessity, been iterative and responsive to the needs of the Institute and its staff. Since its inception 5 years ago, data collection systems have been refined, the methodology has been adjusted to capture appropriate data, staff awareness and participation has increased, and issues regarding the methodology and scoring have been resolved. Conclusions.
The Research Performance Evaluation methodology described here provides a fair and transparent means of disbursing internal funding. It is also a powerful tool for evaluating the Institute’s progress towards achieving its strategic goals, and is therefore a key driver for research excellence. What is known about the topic?
Increasingly, research funders are seeking to evaluate the impact and outcomes of research spending in order to inform policy decisions and guide research funding expenditure. However, in most instances, research evaluation activities are not undertaken by the organisation conducting the actual research and may not meet their practical needs. What does this paper add?
The paper outlines a research performance evaluation methodology specifically tailored to the needs of the medical research institute conducting the research being evaluated, as a way of evaluating research performance to drive strategic goals and disburse internal funds. What are the implications for practitioners?
This paper provides a clear approach to internal research evaluation using a process that meets the needs of the organisation actually conducting the research, and provides institutional data for strategic planning activities.
|
References
[1]
Frank C, Nason E.
Health research: measuring the social, health and economic benefits.
CMAJ 2009
; 180: 528–34.
|
CrossRef |
[2]
Watts G.
Beyond the impact factor.
BMJ 2009
; 338: 440–441.
|
CrossRef |
[3]
Australian Research Council. The Excellence in Research for Australia (ERA) initiative. Canberra: Commonwealth of Australia; 2010. Available at
http://www.arc.gov.au/era/default.htm [verified 8 May 2012].
[4]
Hanney SR, Grant J, Wooding S, Buxton MJ.
Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK’s ‘Arthritis Research Campaign’.
Health Res Policy Syst 2004
; 2: .
|
CrossRef |
[5]
Hanney S, Packwood T, Buxton M.
Evaluating the benefits from health research and development centres: a categorization, a model and examples of application.
Evaluation 2000
; 6: 137–160.
|
CrossRef |
[6]
National Health and Medical Research Council. NHMRC research funding dataset 2000–2009. Canberra: Commonwealth of Australia; 2010. Available at
http://www.nhmrc.gov.au/grants/dataset/rmis/index.htm [verified 8 May 2012].
[7]
Weingart P.
Impact of bibliometrics upon the science system: inadvertent consequences?
Scientometrics 2005
; 62: 117–31.
|
CrossRef |
CAS |
[8]
Wooding S, Hanney S, Buxton M, Grant J. The returns from arthritis research. Leiden: RAND Europe; 2004. Available at
http://www.rand.org/pubs/monographs/2004/RAND_MG251.pdf [verified 8 May 2012].
[9]
Wooding S, Hanney S, Buxton M, Grant J.
Payback arising from research funding: evaluation of the Arthritis Research Campaign.
Rheumatology 2005
; 44: 1145–56.
|
CrossRef |
CAS |
[10]
Wells R, Whitworth J.
Commentary. Assessing outcomes of health and medical research: do we measure what counts or count what we can measure?
Aust New Zealand Health Policy 2007
; 4: .
|
CrossRef |
[11]
Brown H.
How impact factors changed medical publishing – and science.
BMJ 2007
; 7593: 561–4.
|
CrossRef |
[12]
Brumback RA.
Worshiping false idols: the impact factor dilemma.
J Child Neurol 2008
; 23: 365–7.
|
CrossRef |
[13]
Hirsch JE.
An index to quantify an individual’s scientific research output.
Proc Natl Acad Sci USA 2005
; 102: 16 569–72.
|
CrossRef |
CAS |
[14]
SCImago. SJR – SCImago Journal & Country Rank. Granada: SCImago; 2007. Available at
http://www.scimagojr.com [verified 8 May 2012].
[15]
Van Noorden R.
A profusion of measures.
Nature 2010
; 465: 864–6.
|
CrossRef |
CAS |
[16]
Panel on Return on Investment in Health Research. Making an impact: apreferred framework and indicators to measure returns on investment in health research. Ottawa, ON, Canada: Canadian Academy of Health Sciences; 2009. Available at
http://www.cahs-acss.ca/wp-content/uploads/2011/09/ROI_FullReport.pdf [verified 8 May 2012].
[17]
Editorial. Assessing assessment.
Nature 2010
; 465: 845.
|
CrossRef |
[18]
Kuruvilla S, Mays N, Pleasant A, Walt G.
Describing the impact of health research: a Research Impact Framework.
BMC Health Serv Res 2006
; 6: .
|
CrossRef |
[19]
Kuruvilla S, Mays N, Walt G.
Describing the impact of health services and policy research.
J. Health Serv Res Policy 2007
; 12: 23–31.
e-Alerts
Connect with AHR
