Skip to Main Content
Library Guides

Library Research Support Services

Responsible Metrics

"Responsible metrics [is] a way of framing appropriate uses of quantitative indicators in the governance, management and assessment of research."   'The Metric Tide', 2015

The concept of responsible metrics stems from attempts to prevent the mis-use of research metrics, leading to incorrect inferences or undesirable consequences. The landscape has largely moved on to address responsible assessment, looking at more holistic behaviours around the application of metrics for research assessment. However, the original principles of responsible metrics still apply to our everyday use of research metrics for insight, monitoring and strategy.

San Francisco Declaration on Research Assessment

The University is a signatory of the San Francisco Declaration on Research Assessment [DORA]. As an institution this means we:

  • Are clear about criteria used for hiring, tenure and promotion decisions, prioritising the content of research output over publication metrics or the venue of publication.
  • Consider the value and impact of all research outputs, including data and software, in addition to publications.
  • Consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice.

For researchers and leads, in addition to the above, DORA also stipulates that we:

  • Use a range of article metrics and indicators on personal/supporting statements, as evidence of the impact of individual published articles and other research outputs.
  • Challenge research assessment practices that rely inappropriately on Journal Impact Factors and promote and teach best practice that focuses on the value and influence of specific research outputs.

DORA

Best practice for use of metrics

It is important to ensure:

  1. That we are using metrics that are appropriate for the entity and use case. We can do this by acknowledging weaknesses of specific metrics and the context of the query into account, using document-level metrics for documents and author-level metrics for authors, and so forth.

  2. We triangulate metrics by using multiple measures or indicators.
    Where they align, this increases trust in the result. Where they do not align then this should prompt further questioning.

  3. We give those being evaluated/measured the chance to ensure that their information is as up to date as possible.
    For example, asking staff to ensure Scopus profiles are up to date before reports are drawn from SciVal.

  4. Metrics alone should not be used for management decisions.
    Expert opinion and peer review/ qualitative information should be consulted alongside any metrics. Seeking context to metrics may alter their relevance.

Not following these principles can lead to incorrect inferences as well as negative impacts on colleagues.

Examples of the wider uptake of responsible metrics and responsible assessment
 

  • cOAlitionS/PlanS has been signed by UKRI, Wellcome Trust and other funders; "The Funders commit that when assessing research outputs during funding decisions they will value the intrinsic merit of the work and not consider the publication channel, its impact factor (or other journal metrics), or the publisher". There are instances of UKRI are beginning to implement this commitment e.g. UKRI's Narrative CV. Wellcome now expect Wellcome-funded organisations to publicly commit to responsible research evaluation.
     
  • UK Reproducibility Network have produced a Common Statement on Responsible Research Evaluation, which includes more detailed advice on the use and application of research metrics.
     
  • For REF2021 a select number of citation metrics were used, but panels were instructed not to look at journal-level metrics and these were not provided.
     
  • Many institutions are joining the 'More Than Our Rank' initiative, demonstrating a commitment to responsible assessment and recognising value beyond university rankings.

Further Resources

Manifestos, principles and campaigns

Reports and guidance