How do you measure and demonstrate your impact?
This was the topic I was asked to address by the Australian Association of Gerontology’s (AAG) Student and Early Career Group in a pre-conference workshop ahead of the 54th AAG conference.
The session organisers were particularly interested in covering how to access, choose and boost research metrics, including Publications, Citations, the H-index, SciVal, Scopus, Altmetrics, Consumer Engagement, Grants, Policy impact…
A fair bit to squeeze into the allotted 30 minutes!
I knew I would be talking to a diverse group – neuroscientists, dieticians, psychologists, allied health practitioners, clinicians. Some fresh from undergrad starting PhDs, others new to graduate research from an established career in their chosen profession, plus postdoctoral researchers with various levels of experience.
I wanted to make my presentation valuable for everyone. Some would know what a FWCI was, and others might not. But amongst the advice about research metrics and impact, I had a message I wanted these students and Early Career Researchers (ECRs) to hear loud and clear…
Be careful who you compare yourself to! ‘Successful’ researchers come in different shapes, sizes, and metrics – and you ALL have important contributions to make (irrespective of your h-index).
Getting clear on what impact is…
For students and ECRs, particularly in some disciplines, it can be easy to think impact is all about publication metrics – citations, Field Weighted Citation Impact (i.e. FWCI), journal rankings, impact factors, percentile rankings and these days altmetric attention scores.
These metrics are useful things; for assessing research quality, choosing where to publish, benchmarking against the expectations in your field, providing evidence of your expertise and the quality of your work.
But it’s important not to let these numbers define you.
For a research career, publications are undeniably important, but they really are just the beginning.
When I quizzed my AAG student and ECR audience about what impact meant to them I was pleased to hear – ”it’s about making a difference” and “having an impact on society”.
This audience was well across the ARC’s definition of research impact “the contribution that research makes to the economy, society, environment or culture, beyond the contribution to academic research”. Or in NHMRC terms, “the effect of research after it has been adopted, adapted for use, or used to inform further research”.
But research impact takes time.
And not only is it generated from a culmination of endeavours across a career, but for many scholars, research impact is just one part of their broader career impact, that includes various contributions across a discipline, field, or profession at expanding (Departmental, Institutional, National & International) levels.
Building your impact, by building a track record
When you’re starting to build something, even a track record, it helps to know the typical (tried and true) structure. While establishing expertise through publications is usually the foundation, there’s a lot of layers between publications and generating research impacts beyond academia.
Common track record measures include:
- Publications and other outputs (their quality and quantity)
- Research income (whether it’s internal, external, which funding body, a grant/contract)
- Student supervision (Hons/Masters/PhD, timeliness, student outcomes/success)
- Mentoring (within your group, within your organisation, externally)
- Collaborations (with individuals, institutions, community, at state, national, and international levels)
- Partnerships (and associated income, commercialisation activities, internships or accreditations, etc.)
- Leadership and service (to your organisation, your field, your profession e.g. through committees, conferences, peer-review, etc.)
- Research impact beyond academia (the reach and significance of the impact)
And just like actual buildings, the final features can be diverse.
Your track record should reflect your intended impact
The actual details of a researcher’s track record are not only going to be different for different disciplines (many HASS disciplines are known to be less citation based than the sciences for a start) but should also be different relative to a researcher’s intended research impact.
A neuroscientist studying a molecular pathway so they can bring a drug to market is likely to have a very different track record (patents, licensing contracts, commercial income, etc) to a neuroscientist studying the same molecular pathway to enable genetic diagnosis and improved treatment choices (sample or data banks, protocols, patient QoL).
My question to the AAG students and ECRs was, what type of measures and metrics are going to tell your impact ‘story’?
To know that of course, the first question to ask is: What impact do you want to have?
Measuring and demonstrating your impact is easier when you know your ‘why’
If impact is an effect, change or benefit, what effect or change do you want to have?
For my AAG audience, ageing was clearly front and centre.
But what about you? Do you want to change knowledge paradigms in your field, or do you want to have specific effects on health or the environment? Do you want to bring a drug to market, change policies, save an endangered species, or something else?
Knowing your why makes it much easier to plan your pathway towards it.
Using an input to impact logic model (see my version adapted from CSIRO’s impact framework below) can not only help turn your vision for change into a plan for impact but provides a framework to build, measure and evidence your track record along the way (particularly helpful when it comes to writing research grants).
From knowing your why, to knowing your worth…
Getting clear on your why can not only help you create a clear map of what you need to do, what stakeholders you need to engage, what outputs and outcomes you want to deliver, but can also help you keep sight of the impacts you are striving to achieve. So that the metrics you accumulate become the measures to define your impact – without letting them define you.
While metrics might be counted in the same way for everyone, the contributions you make, in the specific manner you make them, driven by your impact goals, and your talents, will be what defines you and they will be like no one else’s.
There are all manner of researchers whose contributions vary from: the discovery and dogma smashers with multiple high profile papers; the methodology or analysis whiz’s who are invited onto everyone’s projects because of their skills; the experimental explorers who creates new ways to do things better than before and then help train others to do it; the science communicators and others with a passion for public outreach; the stakeholder strategists who are networked with all the right people; practice educators who translate research into learning; the clinician researchers; the commercialisation specialists and the many more who are a combination of any of the above (or more).
Whether you’re just starting your graduate research journey now, or are somewhere further down the track figuring out your next steps, when it comes to measuring and demonstrating your impact, first make sure you know your worth:
No one is like you and that is your greatest advantage.
Next week…. how to show your worth.
Photo by Isaac Smith on Unsplash