Research Impact Assessment

When measuring research impact, it is important to define terms and distinguish between stages involved in the research process. Research impact refers to evidence for the application of knowledge making a difference, e.g. to health status, quality of life or cost effectiveness. Knowledge translation essentially refers to the review, synthesis, assessment and application of knowledge by key stakeholders to improve health status or quality of life (Graham et al., 2006).

Research impact is linked to research utilisation, the process for implementing ideas. This can be distinguished from knowledge transfer which refers to collecting and sharing ideas, knowledge and research results to enable new products/services. Knowledge exchange also involves ‘collaborative problem solving between researchers and decision-makers.

To arrive at ‘research impact’, a ‘knowledge to action’ process should ideally be in place. Such a cycle is analogous to ‘reflexive practice’. Stages of Graham’s (2006) ‘knowledge to action’ cycle include:
1) Problem identification and the review of appropriate knowledge to address the problem.
2) Adapting knowledge to local contexts, assessing barriers to knowledge and tailoring and
implementing interventions.
3) Monitoring knowledge use (collecting evidence on who used the knowledge and how, e.g. policy change or guideline development).
4) Evaluating outcomes (the impact of using the knowledge).
5) Sustaining knowledge use (applying the stages to ongoing use of the knowledge).

An action research cycle (involving an ongoing cycle of planning, acting, observing, and critically reflecting) aligns to this process. Monitoring and reflecting upon knowledge use and research utilisation (the process for implementation of ideas) should be an important part of this action research cycle.

The measurement of ‘project spread’ is also an important consideration when considering research impact. The term ‘spread’ is drawn from the Institute for Healthcare Improvement (IHI) Framework (2006).

A key requirement identified within this framework is to assess whether
improvements have been spread within or across the selected projects from the perspective of:
• Leadership, which incorporates executive leaders ensuring that the ‘spread’ aim is aligned with the broader organisational direction and assign day to day responsibility for the spread of processes to a team.
• Setup for spread, which relates to developing the infrastructure to support broader adoption of change, including:
o Engagement of key figures or partners (internal and external) to support and promote
participation in and adoption of changes
o Identification of successful sites from which to draw support, input and learning
o Identification of those likely to readily adopt changes
o Determination of the population to be targeted; and establishment of a plan for attracting those likely to readily adopt the changes

• Marketing change, relates to the development of a case for adopting the proposed changes and to clear describe the changes (or better ideas to use IHI terminology)
• Communication, ensuring that those responsible for leading spread and identified key individuals use appropriate communication to promote awareness of the changes or improvements
• Social system, providing an environment that supports adoption of change through provision of appropriate technical support, identification and response to barriers and developing and using individuals to continue promotion of the changes
• Knowledge management, collecting and responding to information about the spread of change or improvement
• Measurement and feedback, collecting and reporting process and outcome indicators to
highlight achievements of the changes and monitor progress in spreading the change within or across organisations.
Utilisation of this framework implies a series of both process and outcome measures.

See full paper: Battams, S, Research Impact Assessment, HOI Discussion Paper