Measuring research culture: are we shifting the dial?

As the research sector continues to invest in research culture, attention is increasingly turning to a new question: has all this work actually made a difference?

Funders, institutions and sector bodies are rightly seeking to understand impact. Programmes are being evaluated, surveys are being commissioned, and new datasets are emerging to track progress.

This is both necessary and overdue. Improving research culture is not just about initiating change. It is about understanding whether that change is meaningful, where it is having most impact, and how it should evolve over time.

Reflecting at this stage does not have to be about judging success or failure. It can be about making better decisions: identifying which approaches are worth scaling, where further investment is needed, and which interventions may be creating more administration than meaningful impact.

Measurement is not neutral

There is no single, settled approach to evaluating research culture.

In many cases, the focus remains on researchers:

• their experiences

• their perceptions of fairness and inclusion

• the behaviours shaping research practice

This can help to surface the experience of academic work, the conditions under which research is conducted and the incentives and disincentives shaping behaviour.

There are also other organisations and professional bodies seeking out the perspectives of their own corners of the sector. These different attempts to measure progress can draw out different understandings of what research culture is.

Bringing in professional and technical roles can broaden the focus to include:

• experiences across the university

• how research systems function

• how work is organised and supported

• how processes shape day-to-day reality

What should change look like?

Alongside questions of perspective sits a second challenge: how we interpret what we find.

Much of the sector’s major investment in research culture is relatively recent. Large-scale funding programmes have only been operating at meaningful scale for a limited number of years, and earlier activity was smaller in scale.

Culture change at this level is cumulative and uneven. It's not realistic to expect immediate, system-wide transformation. More often, it shows up as:

• new conversations and expectations

• pilot initiatives and local experimentation

• gradual shifts in practice

• early adopters shifting behaviour

These are important signals but they are not the same as fully embedded change.

I think some will be legitimately concerned about the risk that we over-interpret early signals as either success or failure, instead of useful insights to define future action.

The view from practice

In our work, we often see the sector at its best. We work alongside proactive teams, in innovative programmes, and colleagues committed to improving research culture.

There is real progress and pockets of transformation where organisations are experimenting and doing things differently. There is no shortage of strong ideas or promising practice.

Sometimes we only manage ripples instead of waves in a sector often slow to change.

System change in research culture is not about isolated successes. It is about whether those successes can be sustained, scaled and embedded. That raises a more difficult question: whether the sector has the patience and the appetite for long-term investment that moves beyond pilots and exemplars, and into the harder work of making change systemic.

An opportunity to build a fuller picture

Taken together, these questions point not just to challenges in how we measure research culture, but to a clear opportunity.

Across the sector, there is now a growing body of activity: funder-led evaluations, institutional surveys, and work by professional bodies exploring different dimensions of research culture. Each of these offers a partial view. But together, they have the potential to provide a richer, more complete understanding of how research culture is evolving.

This is not a new insight. The 2020 ARMA Research Culture Survey highlighted strong agreement that research culture is a shared responsibility across the research ecosystem, spanning researchers, research managers, technical staff, professional services and institutions themselves.

If responsibility is shared, then understanding and measuring research culture needs to be shared too.

As these processes progress in tandem, there is an opportunity:

• to bring together insights from researchers and professional roles

• to connect lived experience with system-level understanding

• to recognise both early signals of change and the longer-term work still to come

This creates a stronger foundation for decision-making.

Reflecting at this stage is not about judging success or failure. It is about making better decisions: understanding which approaches are worth scaling, where further investment is needed, and which interventions may be creating more administration than meaningful impact.

Because ultimately, measuring research culture well is not just about what we count. It is about using what we learn to accelerate the change we want to see.