CSIRO Publishing blank image blank image blank image blank imageBooksblank image blank image blank image blank imageJournalsblank image blank image blank image blank imageAbout Usblank image blank image blank image blank imageShopping Cartblank image blank image blank image You are here: Journals > Australian Health Review   
Australian Health Review
  Journal of the Australian Healthcare & Hospitals Association
blank image Search
blank image blank image
blank image
  Advanced Search

Journal Home
About the Journal
Editorial Board
For Advertisers
Online Early
Current Issue
Just Accepted
All Issues
Sample Issue
For Authors
General Information
Instructions to Authors
Submit Article
Open Access
For Referees
Referee Guidelines
Review an Article
Annual Referee Index
Call for Reviewers
For Subscribers
Subscription Prices
Customer Service
Print Publication Dates

blue arrow e-Alerts
blank image
Subscribe to our Email Alert or RSS feeds for the latest journal papers.

red arrow Connect with AHR
blank image
facebook   TwitterIcon

red arrow Connect with CP
blank image
facebook twitter youtube


Article << Previous     |     Next >>   Contents Vol 36(2)

Research performance evaluation: the experience of an independent medical research institute

Catherine C. Schapper A , Terence Dwyer A D , Geoffrey W. Tregear B , MaryAnne Aitken A and Moira A. Clay C

A Murdoch Childrens Research Institute, Royal Children’s Hospital, Parkville, VIC 3052, Australia. Email: cathy.schapper@ranzcp.org; maryanne.aitken@mcri.edu.au
B Florey Neuroscience Institutes, Howard Florey Institute, Level 2, Alan Gilbert Building, 161 Barry St, Carlton South, VIC 3053, Australia. Email: geoffrey.tregear@florey.edu.au
C Telethon Institute for Child Health Research, PO Box 855, West Perth, WA 6872, Australia. Email: mclay@ichr.uwa.edu.au
D Corresponding author. Email: terry.dwyer@mcri.edu.au

Australian Health Review 36(2) 218-223 http://dx.doi.org/10.1071/AH11057
Submitted: 10 June 2011  Accepted: 22 September 2011   Published: 25 May 2012

PDF (114 KB) $25
 Export Citation

Background. Evaluation of the social and economic outcomes of health research funding is an area of intense interest and debate. Typically, approaches have sought to assess the impact of research funding by medical charities or regional government bodies. Independent research institutes have a similar need for accountability in investment decisions but have different objectives and funding, thus the existing approaches are not appropriate.

Methods. An evaluation methodology using eight indicators was developed to assess research performance across three broad categories: knowledge creation; inputs to research; and commercial, clinical and public health outcomes. The evaluation approach was designed to provide a balanced assessment across laboratory, clinical and public health research.

Results and discussion. With a diverse research agenda supported by a large number of researchers, the Research Performance Evaluation process at the Murdoch Childrens Research Institute has, by necessity, been iterative and responsive to the needs of the Institute and its staff. Since its inception 5 years ago, data collection systems have been refined, the methodology has been adjusted to capture appropriate data, staff awareness and participation has increased, and issues regarding the methodology and scoring have been resolved.

Conclusions. The Research Performance Evaluation methodology described here provides a fair and transparent means of disbursing internal funding. It is also a powerful tool for evaluating the Institute’s progress towards achieving its strategic goals, and is therefore a key driver for research excellence.

What is known about the topic? Increasingly, research funders are seeking to evaluate the impact and outcomes of research spending in order to inform policy decisions and guide research funding expenditure. However, in most instances, research evaluation activities are not undertaken by the organisation conducting the actual research and may not meet their practical needs.

What does this paper add? The paper outlines a research performance evaluation methodology specifically tailored to the needs of the medical research institute conducting the research being evaluated, as a way of evaluating research performance to drive strategic goals and disburse internal funds.

What are the implications for practitioners? This paper provides a clear approach to internal research evaluation using a process that meets the needs of the organisation actually conducting the research, and provides institutional data for strategic planning activities.


[1]  Frank C, Nason E. Health research: measuring the social, health and economic benefits. CMAJ 2009; 180: 528–34.
CrossRef |

[2]  Watts G. Beyond the impact factor. BMJ 2009; 338: 440–441.
CrossRef |

[3]  Australian Research Council. The Excellence in Research for Australia (ERA) initiative. Canberra: Commonwealth of Australia; 2010. Available at http://www.arc.gov.au/era/default.htm [verified 8 May 2012].

[4]  Hanney SR, Grant J, Wooding S, Buxton MJ. Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK’s ‘Arthritis Research Campaign’. Health Res Policy Syst 2004; 2: .
CrossRef |

[5]  Hanney S, Packwood T, Buxton M. Evaluating the benefits from health research and development centres: a categorization, a model and examples of application. Evaluation 2000; 6: 137–160.
CrossRef |

[6]  National Health and Medical Research Council. NHMRC research funding dataset 2000–2009. Canberra: Commonwealth of Australia; 2010. Available at http://www.nhmrc.gov.au/grants/dataset/rmis/index.htm [verified 8 May 2012].

[7]  Weingart P. Impact of bibliometrics upon the science system: inadvertent consequences? Scientometrics 2005; 62: 117–31.
CrossRef | CAS |

[8]  Wooding S, Hanney S, Buxton M, Grant J. The returns from arthritis research. Leiden: RAND Europe; 2004. Available at http://www.rand.org/pubs/monographs/2004/RAND_MG251.pdf [verified 8 May 2012].

[9]  Wooding S, Hanney S, Buxton M, Grant J. Payback arising from research funding: evaluation of the Arthritis Research Campaign. Rheumatology 2005; 44: 1145–56.
CrossRef | CAS |

[10]  Wells R, Whitworth J. Commentary. Assessing outcomes of health and medical research: do we measure what counts or count what we can measure? Aust New Zealand Health Policy 2007; 4: .
CrossRef |

[11]  Brown H. How impact factors changed medical publishing – and science. BMJ 2007; 7593: 561–4.
CrossRef |

[12]  Brumback RA. Worshiping false idols: the impact factor dilemma. J Child Neurol 2008; 23: 365–7.
CrossRef |

[13]  Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA 2005; 102: 16 569–72.
CrossRef | CAS |

[14]  SCImago. SJR – SCImago Journal & Country Rank. Granada: SCImago; 2007. Available at http://www.scimagojr.com [verified 8 May 2012].

[15]  Van Noorden R. A profusion of measures. Nature 2010; 465: 864–6.
CrossRef | CAS |

[16]  Panel on Return on Investment in Health Research. Making an impact: apreferred framework and indicators to measure returns on investment in health research. Ottawa, ON, Canada: Canadian Academy of Health Sciences; 2009. Available at http://www.cahs-acss.ca/wp-content/uploads/2011/09/ROI_FullReport.pdf [verified 8 May 2012].

[17]  Editorial. Assessing assessment. Nature 2010; 465: 845.
CrossRef |

[18]  Kuruvilla S, Mays N, Pleasant A, Walt G. Describing the impact of health research: a Research Impact Framework. BMC Health Serv Res 2006; 6: .
CrossRef |

[19]  Kuruvilla S, Mays N, Walt G. Describing the impact of health services and policy research. J. Health Serv Res Policy 2007; 12: 23–31.

Subscriber Login


Legal & Privacy | Contact Us | Help


© CSIRO 1996-2014