Dr Elizabeth M Morrow
Research Series 5.1, April 2016
Download the Research
The impact agenda is increasingly central to universities’ research strategies and is moving at pace. Whilst the term ‘impact’ is bandied about, and we think we know what it means, there is little consensus in the sector about how impact is achieved, what kind of organisational conditions support and nurture it, or what success looks like.
The impact case study database from the 2014 Research Excellence Framework (REF) provides, for the first time, a landmark resource for learning about research impact, which this Leadership Foundation for Higher Education study exploits in a "deep dive" focusing on leadership, governance and management (LGM) research. The study set out to gain a conceptual handle on LGM research impact in terms of the processes and mechanisms involved. In particular, we were interested in mapping out how LGM research in these case studies led to change, made a difference or had impact. The results will help leaders and researchers in universities think through issues of impact: what the rationale is for understanding impact, why impact is important, and how it can be shown and maximised in its different contexts. The report provides practical advice for the academic community about understanding complexity and multifaceted non linear routes to impact. In particular, the Adaptive Systems Framework for Advancing Research (AS-FAR) will be useful for researchers in their impact planning and reporting. It can contribute to the development of pathways to impact.
The Leadership Foundation envisages that this research report will add value to the important work universities are doing to position, plan and develop institutional capacity for impact. It will also help inform activities across the higher education sector to enhance the impact of LGM research in the future. Alongside this report by Elizabeth Morrow, we commissioned a report by Professor Sue Dopson and colleagues which reviews available literature on the impact of leadership development interventions in UK higher education and their effectiveness. We hope that these reports will be read and used together as they "stake out the ground" and illuminate what we mean by impact from multiple perspectives. Each is frank about the challenges of measurement and offers a conceptual model of the routes to impact from research as well as questions for future research.
Professor Fiona Ross CBE
Director of Research
Leadership Foundation for Higher Education
Higher education in the UK is currently facing some considerable challenges to show and improve its educational, research, social and economic impact. The unfolding of the impact agenda places higher education under the external gaze of political and public scrutiny. At the same time, the impact agenda offers a lens through which the higher education sector might see opportunities for change from individual teaching or research endeavours, to impact at departmental or institutional levels, and across the sector as a whole1. The diverse interests, ideologies and complex politics associated with impact mean there is a need to build a theoretically informed evidence base to guide the practice and assessment of different areas of higher education impact.
The study presented here explored the impact of higher education leadership, management and governance (LGM) research. Leadership, governance and management are diverse fields of practice that are multidisciplinary and cross sectoral2. As large, complex public sector institutions such as universities, local government and the NHS search for evidence-based approaches to improve leadership3, performance4 and competitiveness5, it is becoming increasingly important to understand how research outcomes have led to change, made a difference or had an impact6.
Evidence of research impact is useful for:
Research impact covers an extensive range of issues about research practice, research management, research assessment, researcher skills and capacity development as well as issues about the types of research outputs that are useful to research users and the outcomes of research for beneficiaries. It is challenging, if not impossible, to unpick the concept to examine any one issue in isolation12. Hence understanding research impact takes time, consideration of different perspectives, and a critical eye to where the focus on research impact falls13.
To understand research impact there is a need to unpack the complex processes of interaction between researchers and nonacademic groups, such as collaborators, stakeholders and research users. This includes exploring the different positions and roles of diverse groups of people and the different types of processes of interaction involved, such as collaboration, engagement and involvement in the research. Public/user involvement in research is now seen as an essential part of many research fields and disciplines, in contrast to seeing end-users as the passive subjects or beneficiaries of research.
This study drew on research information returned by higher education institutions (HEIs) in the 2014 Research Excellence Framework (REF) assessment14. For the first time, the 2014 REF included structured impact case studies, which were used to assess the impact of research beyond academia. Assessment of impact was based on expert review of structured impact case studies for the assessment period 1 January 2008 to 31 July 2013. Impact case studies were assessed against the broad generic criteria of the ‘reach and significance’ of the impact or benefit. Assessment of impact was 20% of the total REF score (with outputs making up 65% and environment 15% of the total score) with the intention of increasing this in subsequent exercises15. In total, 6,975 impact case studies were submitted by 154 universities that were assessed. Of these, 6,679 case studies have been made publicly available on a searchable web-based database http://impact.ref.ac.uk/CaseStudies/.
1 Carter, I. (2013). The impact of impact on universities: skills, resources and organisational structures, in Dean, A. et al (eds) 7 Essays on Impact. DESCRIBE Project Report for Jisc. University of Exeter.
2 Bolden, R., Gosling, J., O’Brien, A., Peters, K., Ryan, M. & Haslam, S. (2012). Academic leadership: Changing conceptions, identities and experiences in UK higher education. London: Leadership Foundation for Higher Education.
3 Hamlin, B. (2010). Evidence-based leadership and management development, in Gold, J., Thorpe, R. & Mumford, A. (eds) Gower Handbook of Leadership and Management Development pp 197-220. Farnham, UK: Gower.
4 Pedersen, D. & Hartley, J. (2008). The changing context of public leadership and management: implications for roles and dynamics. International Journal of Public Sector Management 21(4): 327-39.
5 Burns, J. (2001). Complexity science and leadership in healthcare. Journal of Nursing Administration 31(10): 474-82.
6 Given, L.M., Kelly, W. & Willson, R. (2015). Bracing for impact: the role of information science in supporting societal research impact, in Proceedings of the 78th ASIS&T Annual Meeting: Information Science with Impact: Research in and for the Community p 48. American Society for Information Science.
7 Bornmann, L. (2013). What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society for Information Science and Technology 64(2): 217-33.
8 Grant, J., Brutscher, P., Kirk, S., Butler, L. & Wooding, S. (2010). Capturing research impacts: a review of international practice. Documented briefing. Rand Corporation.
9 Donovan, C. (2011). State of the art in assessing research impact: introduction to a special issue. Research Evaluation 20(3): 175-79.
10 LSE Public Policy Group (2011). Maximising the impacts of your research: A handbook for social scientists. London: London School of Economics and Political Science.
11 Upton, S., Vallance, P. & Goddard, J. (2014). From outcomes to process: evidence for a new approach to research impact assessment. Research Evaluation 23(4): 352-65.
12 Brewer, J.D. (2011). The impact of impact. Research Evaluation 20(3): 255-56.
13 Penfield, T., Baker, M., Scoble, R. & Wykes, M. (2014). Assessment, evaluations, and definitions of research impact: a review. Research Evaluation 23(1): 21-32.
14 Hefce (2011). Research Excellence Framework 2014. Assessment framework and guidance on submissions. July 2011. Bristol: Higher Education Funding Council for England.
15 Hefce (2015). REF Accountability Review. Bristol: Higher Education Funding Council for England.
Hard copies available
Our research is a membership benefit and is distributed to all member institutions. If you are an LF member and would like a hard copy publication we are happy to send it to you. There will be a small postage fee of £6.99, for more information, click here.
If you are interested in learning more about being a member and the benefits read more.
This publication can be ordered by non-members for a nominal cost of £20.00 which includes delivery costs. For more information, click here.
Are you interested in Research Leadership Development?
Click here to find out what we can offer you and your team.