Applying an inclusion lens to literature reviewing and research question development

Because many health and social care researchers have only more recently become aware of the importance of inclusive research, much of the literature that informs our proposed research is based on designs and sampling strategies that have traditionally yielded predominantly white, male, well-educated samples. Many studies do not report on the ethnicity, socio-economic status and other demographic characteristics of their samples (Vyas et al. 2018; Furler et al. 2012). Of 1518 COVID-19 trials registered on ClinicalTrials.gov by June 2020, only six were collecting data about ethnicity – despite the disproportionately severe impacts of Covid-19 on Black and South Asian people (Treweek et al. 2020).

The needs and perspectives of specific groups remain invisible when they are either not participants in research or are not disaggregated in the analysis (Petkovic et al. 2020; Phillips and Hamberg 2016). This prevents us from knowing whether different groups have different needs or perspectives, or whether an intervention or new service is safe and/or effective for all population groups. Yet, changes to policy and practice are commonly implemented for the whole population based on this partial evidence. This contributes to certain groups of the population being under-served; that is, less likely to be included in research, more likely to experience a higher burden of health and/or social care need, and likely to have different experiences and/or responses to new interventions compared to other groups  (NIHR INCLUDE guidance). With research inclusion now recognised as integral to all research, we need to critically reflect on the implications of knowledge gaps about the most under-served groups and avoid reproducing these limitations (Witham et al. 2020).

When describing your research problem, be clear about which groups of the population are affected or where there might be evidence gaps for certain groups.

Historical exclusion and exploitation in research

As researchers, we also need to understand that historic and ongoing experiences of oppression and exploitation can damage trust in health, social care and research professionals.

In contemporary times, statistics from Public Health England (PHE) about uptake of the Covid-19 vaccine in England indicate that amongst the over 50s, 95.4% of White British and 93.6% of Indian people had received two doses by October 2021. This was compared to 70.1% of Black Caribbean and 78.1% of Black African people (PHE 2021). Qualitative research conducted by ARC East Midlands (2020) revealed the concerns which different minority and/or marginalised groups have about participating in vaccine trials. Black African and African-Caribbean communities were most suspicious about the vaccine and questioned whether there was a hidden agenda to use Black people as ‘guinea pigs’ or even seek to eradicate them.

These suspicions need to be understood in the context of ethical atrocities and long-standing health and care inequalities which signal that Black people’s lives have less worth. For example, from univariate analysis from mortality reports, Black women have more than a four-fold risk of dying during childbirth (Knight et al. 2020) and Black people are almost four times more likely than white people to be sectioned under the Mental Health Act (NHS 2020). Many other groups have experienced exploitation or exclusion. Girls and women have been under-diagnosed with autism and not given appropriate care and support due to assumptions and diagnostics being based on how autistic boys and men present  (Lockwood Estrin et al. 2020). Lesbian, gay, bisexual, transgender, queer and intersex (LGBTQI+) people may have been refused care or subjected to harmful and demeaning treatments (Vincent, 2018; Smith et al. 2004).

Structural inequality and injustices lead to mistrust and fear of health and social care professionals, and of researchers. This is exacerbated by under-served communities having limited understanding of research because they are seldom offered opportunities to participate. Sensitive and committed outreach, relationship-building and public involvement will be required to build trust and encourage participation (Prinjha et al. 2020; Hussain-Gambles 2004). Keeping to your promises is vital when working with groups that have been repeatedly exploited and disrespected, as well as being clear and realistic about what the benefits of research participation will be.

Key messages

• Being aware of excluded or under-served groups in both health and care inequalities and representation in research will enable you to align your research questions and design to where the need is greatest.
• Understanding different communities’ previous experiences within health, social care and research can enable you to plan appropriate time, resources and staffing to engage effectively with groups that need to include in your research.

• Health Equity in England: The Marmot Review 10 Years On provides a very useful overview of the health of people in England, including data on regional differences and socio-economic inequalities.
• NIHR ARC North West Coast’s Health Inequalities Assessment Toolkit (HIAT) can be used by researchers (as well as clinicians, managers and commissioners) to consider how their proposed work will tackle health inequalities
• Diversity and inclusion in health services research – a 90 minute panel discussion recorded for the Health Services Research UK (HSR-UK) 2020 conferenc

ARC East Midlands (2020) Public perceptions towards vaccine trial research within ethnic minority and vulnerable communities, Leicester: ARC East Midlands.

Furler, J. et al. (2012) ‘Participant demographics reported in "Table 1" of randomised controlled trials: a case of "inverse evidence"?’, International Journal for Equity in Health, 11: 14.

Hussain‐Gambles, M., Atkin, K. and Leese, B. (2004) ‘Why ethnic minority groups are under‐represented in clinical trials: a review of the literature’, Health & Social Care in the Community, 12: 382-388.

Knight et al. on behalf of MBRRACE-UK (2020) Saving Lives, Improving Mothers’ Care: Lessons learned to inform maternity care from the UK and Ireland Confidential Enquiries into Maternal Deaths and Morbidity 2016-18, Oxford: National Perinatal Epidemiology Unit, University of Oxford. Available online at: https://www.npeu.ox.ac.uk/assets/downloads/mbrrace-uk/reports/maternal-report-2020/MBRRACE-UK_Maternal_Report_Dec_2020_v10_ONLINE_VERSION_1404.pdf (accessed 27/04/21).

NHS (2020) ‘Mental Health Act Statistics, Annual Figures 2019-20’. Available online at: https://digital.nhs.uk/data-and-information/publications/statistical/mental-health-act-statistics-annual-figures/2019-20-annual-figures (accessed 27/04/21).

Petkovic, J. et al. (2020) ‘Reporting of health equity considerations in cluster and individually randomized trials’, Trials, 21: 308.

Phillips, S.P. and Hamberg, K. (2016) ‘Doubly blind: a systematic review of gender in randomised controlled trials’, Global Health Action, 9:1.

Prinjha, S. et al. (2020) ‘Including ‘seldom heard’ views in research: opportunities, challenges and recommendations from focus groups with British South Asian people with type 2 diabetes’, BMC Medical Research Methodology, 20: 157.

Public Health England (PHE) (2021) COVID-19 Health Inequalities Monitoring for England (CHIME) tool. Available online at: https://analytics.phe.gov.uk/apps/chime/ (accessed 19/01/22).

Reverby, S. (2010) ‘A new lesson for the old “Tuskegee” Study’. Available online at: https://www.huffpost.com/entry/a-new-lesson-from-the-old_b_378649 (accessed 19/01/22).

Smith, G., Bartlett, A. and King, M. (2004) ‘Treatments of homosexuality in Britain since the 1950s – an oral history: the experiences of patients’, BMJ, doi:10.1136/bmj.37984.442419.EE  

Treweek, S. et al. (2020) ‘COVID-19 and ethnicity: who will research results apply to?’, The Lancet, 395: 1955-1957.

Vincent, B.W. (2018) ‘Studying trans: recommendations for ethical recruitment and collaboration with transgender participants in academic research’, Psychology & Sexuality, 9(2): 102-116.

Vyas, M.V. et al. (2018) ‘Representation of ethnic groups in dementia trials: systematic review and meta-analysis’, Journal of the Neurological Sciences, 394: 107-111.

Witham, M. et al. (2020) ‘Developing a roadmap to improve trial delivery for under-served groups: results from a UK multi-stakeholder process’, Trials, 21: 694.

Embedding inclusion at the design stage will make your research more accessible and result in higher quality, more impactful and cost-effective research. This will increase the likelihood of your study recruiting sufficient participants, achieving diverse participation and keeping to its timelines. It also signals to prospective participants that you are considerate of their needs and preferences. This is particularly important when trying to recruit and include groups of the population that are under-served. It is common to see grant applicants making statements about their intentions to recruit a diverse sample, but a reviewer may ask, ‘Which measures will be taken to make the research inclusive, and what expertise does the team already have in conducting research with diverse and under-represented populations?’.

Many studies have examined barriers to the participation of under-represented and minority groups in applied health and social care research. The greatest focus is on how to more effectively include Black, Asian and minority ethnic communities (Prinjha et al. 2020; Ahmed et al. 2019; Lloyd et al. 2008; Hussain-Gambles et al. 2004). Useful resources include:

• the Centre for Ethnic Health Research’s (2018) toolkit for increasing the participation of Black, Asian and minority ethnic communities in health and social care research and
• the INCLUDE Ethnicity Framework

These resources highlight the importance of tailoring approaches to including different communities through being aware of different linguistic, communication and practical needs, and differing levels of health literacy and awareness of health and social care research. They also emphasise the importance of building trust, developing reciprocal relationships, and developing strong links with relevant contacts and community organisations.

Researchers need to have good cultural awareness and knowledge e.g. being familiar with common foods that are commonly eaten within different ethnic communities in research that asks questions about diet, as Prinjha et al. (2020) advise. Some research topics or questions might be culturally, religiously or politically sensitive amongst different communities. Medical conditions, treatments, social care needs and healthcare experiences that researchers may feel very at ease with discussing may be very controversial or offensive to tackle without first building sufficient trust and understanding (e.g. Culley et al. 2007 in relation to South Asian communities and infertility).

Other issues to consider are whether participants would prefer to engage with researchers who are either from inside or outside of their communities (Vickers et al. 2012), and for data collection activities to be differentiated by sex, age or faith (Centre for Ethnic Health Research 2018). Early patient and public involvement is essential to guide your decisions and it is important to avoid homogenizing the preferences of communities and remember that there are often as many differences within communities as there are between them (Riggs and das Nair 2012).

Language and communication are key issues. There are significant scientific limitations if you exclude prospective participants who cannot give informed consent in English. Often, therefore, it will be necessary to allow sufficient budget and time for translation of written materials and/or for the use of interpreters (see ‘Budgeting for Inclusion’ domain). There are various issues to consider with translation and the use of interpreters, whether for translating outcome measures, seeking informed consent or conducting qualitative interviews via an interpreter.

Barriers related to communication are not unique to minority ethnic communities. Wider questions need to be asked about how accessible your research is to those with low literacy, and those who do not have easy or any access to the internet:

• If prospective participants need to have an email address in order to be recruited, whose voices and experiences will be omitted?
• Can opportunities to participate be communicated in other ways?
• How should communication (written and verbal) take neurodiversity into consideration?

Accessibility also requires consideration:

• How far will participants have to travel to take part in the study, and what support is being given to those to who live in rural areas, who are disabled, or who are carers?
• Is it possible for some testing or follow-up monitoring to take place in general practices or other community facilities or remotely, rather than in hospitals?
• Can participants choose from a range of times to attend appointments for either an intervention or data collection to fit around work, care commitments and the availability of informal carers? Can child care facilities be made available?
• Can methods or data collection protocols be amended to enable those with physical or cognitive processing impairments to participate, and to ensure informed consent?

Thinking through these issues at the design stage is critical because they may have cost implications.

A growing literature has also shared learning and recommendations for conducting health or social care research with various under-served groups, not exhaustively:

Although various adaptations can be made to conduct research to be inclusive of different needs, it might be necessary to consider whether conventional methods are going to be most effective for the population(s) that you are studying. Conventional methods such as questionnaires, interviews and focus groups may not be very accessible to those with low literacy, social anxiety or cognitive impairment  (Fearon 2019; Phillipson and Hammond 2018). They may also feel overly formal or objectifying to research participants. When trying to reach a community that feels disempowered, under-valued or misrepresented by health and social care professionals and researchers, participatory approaches such as participatory action research may help to redress power imbalances and share knowledge and skills (Baum et al. 2006; Cargo and Mercer 2008). More novel and creative approaches could also be used, such as:

• photovoice (Drainoni et al. 2019; Cluley 2017)
• photo elicitation interviews (Fearon 2019)
• arts-based methods (Bird 2017; Coemans and Hannes 2017)

These methodologies are less dependent on verbal communication and can offer insights which are multi-sensory and may be a less threatening and more empowering means of exploring sensitive topics.

NIHR are aware that changes to data collection methods to improve the inclusivity of the study may result in additional costs. As of 27 November 2024, reasonable and justified costs associated with these changes will be covered by NIHR and will not affect the study’s value for money.

Key messages

• It is important to scrutinise each step of the data collection process for barriers to participating that could affect participant recruitment, retention, experience and the quality of the data that you will collect.
• Creative and innovative methods such as arts-based methods and photovoice may overcome some communication barriers and yield rich data in a way which is also more empowering for participants.

• Autism@Manchester’s guidelines for conducting research with the autistic community
• Centre for Ethnic Health Research’s toolkit for increasing the participation of Black, Asian and minority ethnic communities in health and social care research
• NIHR (2024) Disability Framework
• Social Change UK – A brief guide to creative research methods
• The INCLUDE Ethnicity Framework is a valuable resource for planning trials and includes prompts for considering what barriers may prevent different ethnic groups from participating in clinical trials, and how to address these.

Ahmed, A. et al. (2019) ‘An examination of how to engage migrants in the research process: building trust through an ‘insider’ perspective’, Ethnicity & Health. Advance online publication.

Baum, F., MacDougall, C. and Smith, D. (2006) ‘Participatory action research’, Journal of Epidemiology and Community Health, 60(10): 854-857.

Bigby, C., Frawley, P. and Ramcharan, P. (2014) ‘Conceptualizing inclusive research with people with intellectual disability’, Journal of Applied Research in Intellectual Disabilities, 27: 3–12.

Bird, J. (2017) ‘Art therapy, arts-based research and transitional stories of domestic violence and abuse’, International Journal of Art Therapy, 23(1): 14-24.

Bonevski, B. et al. (2014) ‘Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups’, BMC Medical Research Methodology, 14: 42.

Brown, P. and Scullion, L. (2010) ‘‘Doing research’ with gypsy-travellers in England: reflections on experience and practice’, Community Development Journal, 45(2): 169-185.

Cargo, M. and Mercer, S.L. (2008) ‘The value and challenges of participatory research: strengthening its practice’, Annual Review of Public Health, 29: 325-350.

Cluley, V. (2017) ‘Using photovoice to include people with profound and multiple learning disabilities in inclusive research’, British Journal of Learning Disabilities, 45(1): 39-46.

Coemans, S. and Hannes, K. (2017) ‘Researchers under the spell of the arts: Two decades of using arts-based methods in community-based inquiry with vulnerable populations’, Educational Research Review, 22: 34-49.

Condon, L. et al. (2019) ‘Engaging Gypsy, Roma, and Traveller Communities in Research: Maximizing Opportunities and Overcoming Challenges’, Qualitative Health Research, 29(9): 1324–1333.

Culley, L., Hudson, N. and Rapport, F. (2007) ‘Using Focus Groups With Minority Ethnic Communities: Researching Infertility in British South Asian Communities’, Qualitative Health Research, 17(1): 102–112.

Drainoni, M.L. et al. (2019) ‘“We don't get much of a voice about anything”: perspectives on photovoice among people who inject drugs’, Harm Reduction Journal, 16: 61.

Fearon, K. (2019) ‘‘Have you ever talked to any women with Turner syndrome?’ Using universal design and photo elicitation interviews in research with women with mild cognitive impairment’, Methodological Innovations, advance online publication.

Frankena, T. K. et al. (2019) ‘A consensus statement on how to conduct inclusive health research’, Journal of Intellectual Disability Research, 63: 1–11.

Gowen, E. et al. (2019) ‘Guidelines for conducting studies with the autism community’, Autism Policy & Practice, 2(1): 29-45.

Hussain‐Gambles, M., Atkin, K. and Leese, B. (2004) ‘Why ethnic minority groups are under‐represented in clinical trials: a review of the literature’, Health & Social Care in the Community, 12: 382-388.

Lloyd, C.E et al. (2008) ‘Securing recruitment and obtaining informed consent in minority ethnic groups in the UK’, BMC Health Services Research, 8: 68.

McMillan, B. et al. (2009) ‘Studying the infant feeding intentions of pregnant women experiencing material deprivation: Methodology of the Looking at Infant Feeding Today (LIFT) study’, Social Science & Medicine, 68(5): 845-849.

McMurdo, M.E.T. et al. (2011) ‘Improving recruitment of older people to research through good practice’, Age and Ageing, 40(6): 659–665.

Nicolaidis, C. et al. (2020) ‘Creating accessible survey Instruments for use with autistic adults and people with intellectual disability: Lessons learned and recommendations’, Autism in Adults, 2(1): 67-76.

Owen-Smith, A.A. et al. (2016) ‘Perceptions of Barriers to and Facilitators of Participation in Health Research Among Transgender People’, Transgender Health, 1(1): 187-196.

Patel, N. et al. (2016) ‘Developing a Conceptually Equivalent Type 2 Diabetes Risk Score for Indian Gujaratis in the UK’, Journal of Diabetes Research, Article ID 8107108.

Phillipson, L. and Hammond, A. (2018) ‘More Than Talking: A Scoping Review of Innovative Approaches to Qualitative Research Involving People With Dementia’, International Journal of Qualitative Methods, 17: 1-13.

Price, K.N. et al. (2020) ‘Facilitating clinical trials participation of low socioeconomic status patients’, Dermatology, 237(5): 843-846.

Prinjha, S. et al. (2020) ‘Including ‘seldom heard’ views in research: opportunities, challenges and recommendations from focus groups with British South Asian people with type 2 diabetes’, BMC Medical Research Methodology, 20: 157.

Riggs, D., & das Nair, R. (2012) ‘Intersecting Identities’ in das Nair, R. and Butler, C. (eds) Intersectionality, Sexuality and Psychological Therapies: Working with Lesbian, Gay and Bisexual Diversity. Oxford: Blackwell Publishing, pp.9-30.

Rios, D. et al. (2016) ‘Conducting accessible research: including people with disabilities in public health, epidemiological, and outcome studies’, American Journal of Public Health, 106(12): 2137-2144.

Squires, A. (2008) ‘Language barriers and qualitative nursing research: methodological considerations’, International Nursing Review, 55: 265-273.

Tanner, C.T. et al. (2018) ‘Conducting research with older adults with vision impairment: Lessons learned and recommended best practices’, Gerontology & Geriatric Medicine, 4: 1-6.

Vickers, T., Craig, G. and Atkin, K. (2012) Research with black and minority ethnic people using social care services, London: NIHR School for Social Care Research.

Vincent, B.W. (2018) ‘Studying trans: recommendations for ethical recruitment and collaboration with transgender participants in academic research’, Psychology & Sexuality, 9(2): 102-116.

Young, A. and Hunt, R. (2011) Research with d/Deaf people. London: NIHR School for Social Care Research.

Zermansky, A.G. et al. (2007) ‘Striving to recruit: the difficulties of conducting clinical research on elderly care home residents’, Journal of the Royal Society of Medicine, 100(6): 258–26.

To develop this guidance, we collaborated with RDS public contributors to find out what they thought that researchers needed to know about inclusion within patient and public involvement (PPI). Their perspectives are informed by their lived experience, different identities and experiences of working with researchers.

The first question, ‘How aware are you of EDI issues in public involvement?’ covers various sub-questions, such as:

Public contributors need to be assured that you have taken time to educate yourself both on research inclusion and on the previous experiences of their communities/groups in research. If you have not yet done so:

In addition to doing this groundwork, you need to decide which model of public involvement is appropriate for your study; for example:

• Consultation
• Collaboration
• Participatory, co-production or user-led approaches

These are characterised by varying levels of involvement: consultation offers the least opportunity for traditional power relations in the research process to be disrupted. In contrast, co-produced, participatory or user-led approaches are potentially the most empowering models of public involvement (see NIHR (2021a) public  involvement briefing note 7;  NIHR (2021b) co-production guidance; Ocloo and Matthews 2015).

Your chosen approach needs to suit your research aims and objectives, but research inclusion is highly relevant too. If you hope to include populations that have historically been marginalised and ‘done to’, a participatory or user-led approach may be more empowering and would share power more equitably (see NIHR (2021c) co-production in action case studies). When considering participatory approaches, do not under-estimate how much time will be involved in developing relationships and shared vision, mutual learning and training, communication and making decisions democratically.

Once you are ready to start planning your public involvement activities, the second question, ‘How will your public involvement strategy enable you to recruit and retain diverse public contributors?’, requires you to consider:

Just as you would usually seek a diverse sample of research participants, perhaps recruiting from multiple sites, you should also consider where you recruit your public contributors from . The public contributors who are most accessible – perhaps those who belong to an existing patient or user involvement group – may lack diversity. A range of lived experience is required to draw out varied perspectives (Ekezie et al. 2021). People from groups or communities who do not readily volunteer to be involved in research may be labelled ‘hard-to-reach’. However, it is more appropriate to describe them as under-served and to scrutinise how we can better reach out to them and involve them.

Varied recruitment strategies will be needed to recruit a diverse public involvement group, such as:

• Attending community events
• Using social media (e.g. Facebook, Twitter, WhatsApp)
• Developing links with places of worship and community venues
• Reaching out to community leaders and activists
• Giving talks on local radio (Centre for Ethnic Health Research 2018)

Often, the most fruitful relationships develop out of sustained community engagement. This is not a quick approach: it takes time to identify the most appropriate contacts, build trust and mutual understanding, and learn about the dynamics, values, and power relations within a particular community or group (Cowan 2020). It is not reasonable to expect one individual to speak for their community; rather, they are sharing their lived experience. Caution should be applied in situations where a community member tries to position themselves as a spokesperson.

Depending on your background and prior experience, developing skills for public involvement and community engagement takes time. To facilitate this work and build your skills, you could include an experienced public involvement lead in your team. NIHR funding programmes require that a co-applicant or named member of the research team should be budgeted for as the PPI lead. Applicants need to show that this individual (or individuals, if the role is shared) has the relevant skills, experience and authority to coordinate the public involvement work (NIHR 2021e). Although it is not a requirement, where engagement and public involvement centres on under-served communities, it would be an asset for the PPI lead to have prior experience of these communities or groups (whether research- or practice-based, or through lived experience). Importantly though, no one should feel obliged or pressured into taking this role.

The third question, ‘How will you support and empower your diverse public involvement group?’, focuses on how to work effectively and equitably with your diverse public involvement group. Questions to consider here are:

Successful public involvement requires transparency, regular dialogue, and actively empowering public contributors to have a meaningful role in your research. It is important to clarify, not only at the outset, but throughout, what is expected of public contributors, particularly if these expectations change. Discussions should take place with and not simply about public contributors on these matters. The research team also needs to set out what public contributors can expect from them, for example, frequency of contact, availability of training and support, and promptness of payments. Opportunities for sustaining relationships with public contributors beyond the lifetime of the project should be sought (Jinks et al., 2016), but manage public contributors’ expectations so that they do not feel ‘dropped’ or forgotten when a project ends.

Building relationships with public contributors is crucial:

• Incorporate opportunities for the research team and public contributors to get to know each other more informally. This will help to break down barriers of ‘them and us’ and fosters empathy and humanity.
• Getting to know your public contributors will help to avoid making prejudgements about their identities, experiences or capabilities.
• Avoid seeing public contributors as fulfilling a quota because they belong to a particular ethnic group, receive a particular type of care, or fall into a given age bracket. Their diversity will come from their overlapping identities and varied lived experience, beyond what is immediately visible.

A key challenge in public involvement concerns power relations. Public contributors may be intimidated by researchers’ qualifications, titles and long lists of publications. There may also be power differentials between public contributors, for example, resulting from different levels of experience of public involvement, or structural factors such as gender, ethnicity and age. Power imbalances can inhibit public contributors from being heard or feeling able to engage. Cowan’s (2020) report on learning from the Reaching Out projects encourages researchers to relinquish power and be prepared to take themselves out of their comfort zones. Some simple steps which can make a difference include:

• Not using titles
• Holding meetings in community venues and not in a boardroom format
• Giving everyone the opportunity to speak
• Ensuring that public contributors are not heavily outnumbered by research team members
• Agreeing ground rules such as not interrupting and offering criticism constructively

In addition, be careful to distinguish between ‘focus groups’ as a qualitative research method and public involvement discussions. public involvement discussions should be freer than focus groups typically are and should not simply reflect the researchers’ agenda. Power dynamics will operate differently when comparing a focus group that is steered by a researcher and is relatively restricted in scope, and less hierarchical public involvement discussions which can stimulate researchers to think about their research in different ways, throughout its whole cycle.

Your ability to recruit and, importantly, retain a diverse public involvement group will depend on how inclusive and accessible your involvement activities and processes are.

Critical reflection upon, and evaluation of, your public involvement activities will help you to identify how you can improve public contributors’ experiences of involvement in your project. This will help to maximise the value and relevance of their contributions (see Gibson et al. 2017; Kok 2018). Issues to reflect on include:

• Are some voices not being heard, and if so, why, and what can you do to empower them?
• Might research inclusion issues account for some public contributors having less positive experiences of your public involvement  activities than others?
• How is public involvement impacting your research? Are you being sufficiently open to all perspectives, particularly those which challenge your own expectations or experiences?

With thanks to Cecily Henry, Pam Rees and Beauty Tshuma who co-developed the questions and the shape and key themes of this section, with the support of Naina Patel, PPI Lead at RDS East Midlands.

Key messages

• Involving and listening to public contributors with diverse identities and lived experience will make your research more relevant and accessible. It is crucial if you want to study groups that have often been excluded from research.
• To conduct inclusive public involvement, you may have to leave your comfort zone to hold meetings in community spaces, join in with community activities and share power more equitably.

• NIHR (2021) Public Involvement in Social Care Research
• NIHR (2024) Disability Framework
• NIHR (2023)  Tips for researchers involving unpaid carers in health and care research
• Volunteering centre websites are a good starting point for finding out about groups and organisations in the area(s) where your study will take place. Find your nearest volunteering centre at https://www.ncvo.org.uk/ncvo-volunteering/find-a-volunteer-centre

Centre for Ethnic Health Research (2018) Toolkit for: Increasing Participation of Black Asian Minority Ethnic (BAME) Groups in Health and Social Care Research. Leicester: ARC East Midlands.

Cowan, K. (2020) A practical guide to being inclusive in public involvement in health research: Lessons learnt from the Reaching Out programme, Southampton: NIHR INVOLVE. Available online at: https://www.learningforinvolvement.org.uk/wp-content/uploads/2021/04/NIHR-Reaching-Out_-A-practical-guide-to-being-inclusive-in-public-involvement-in-health-research-Lessons-learnt-from-the-Reaching-Out-programme-April-2021.pdf (accessed 22/04/21). (For more information about the Reaching Out projects, visit https://www.invo.org.uk/current-work/reaching-out/?print=print).

Ekezie, W. et al. (2021) ‘Patient and public involvement for ethnic minority research: an urgent need for improvement’, Journal of the Royal Society of Medicine. Advance online publication.

Gibson, A., Welsman, J. and Britten, N. (2017) ‘Evaluating patient and public involvement in health research: from theoretical model to practical workshop’, Health Expectations, 20(5): 826-835.

Jinks, C. et al. (2016) ‘Patient and public involvement in primary care research - an example of ensuring its sustainability’, Research Involvement and Engagement, 2: 1.

Kok, M. (2018) Guidance Document: Evaluating public involvement in research, Bristol: UWE Bristol. Available online at: http://www.phwe.org.uk/wp-content/uploads/Guidance-on-evaluating-Public-Involvement-in-research.pdf (accessed 5/05/21).

NIHR (2021a) Briefing note seven: approaches to public involvement in research. Available online at: https://www.nihr.ac.uk/documents/briefing-notes-for-researchers-public-involvement-in-nhs-health-and-social-care-research/27371#Briefing_note_seven:_approaches_to_public_involvement_in_research (accessed 14/04/21).

NIHR (2021b) Guidance on co-producing a research project. Available online at: https://www.learningforinvolvement.org.uk/wp-content/uploads/2021/04/NIHR-Guidance-on-co-producing-a-research-project-April-2021.pdf (accessed 14/04/21).

NIHR (2021c) Co-production in action: number one. Available online at: https://www.learningforinvolvement.org.uk/wp-content/uploads/2021/04/NIHR-Co-production-in-Action-Number-One-April-2021.pdf (accessed 14/04/21).

NIHR (2021d) Payment guidance for researchers and professionals. Available online at: https://www.nihr.ac.uk/documents/payment-guidance-for-researchers-and-professionals/27392 (accessed 14/04/21).

NIHR (2021e) Definition and role of the designated PPI lead in a research team. Available online at: https://www.nihr.ac.uk/documents/definition-and-role-of-the-designated-ppi-patient-and-public-involvement-lead-in-a-research-team/23441 (accessed 25/10/2021).

NIHR RDS South East and NIHR Centre for Engagement and Dissemination (2020-2021) That co-production podcast!. Available online at: https://www.rds-se.nihr.ac.uk/podcasts/ (accessed 14/04/21).

Ocloo, J. and Matthews, R. (2015) ‘From tokenism to empowerment: progressing patient and public involvement in healthcare improvement’, BMJ Quality & Safety, 25: 626-632.

Vincent, B.W. (2018) ‘Studying trans: recommendations for ethical recruitment and collaboration with transgender participants in academic research’, Psychology & Sexuality, 9(2): 102-116.

How will you analyse and report EDI characteristics?

In any research project you will have a main analysis which will be carried out on your research sample.  Statistics on participant characteristics (demographics) are obtained from your sample to describe your target population e.g. age, sex, gender and ethnicity. Your research sample should reflect the population that you want to implement the research findings on. This might not necessarily reflect the whole of the UK e.g. your disease area or social care need might be more common in older people such as Chronic Obstructive Pulmonary Disease (COPD) or Parkinson’s Disease, so your sample would be older.

When it comes to research analysis, it’s very common to see participants’ demographic data reported on in ‘Table 1’. This reporting may be because questions of demographic differences are not considered central to the study, and it may also reflect some of the statistical debates related to undertaking subgroup analyses, and the potential misuse of these.

Within your sample though there may be effects that are dependent on demographics that can be examined. We can test these formally through interaction effects in statistical analysis, but these are often underpowered due to not being statistically considered for such analysis. This means that minoritised groups typically become indistinguishable from the majority in statistical analysis, and the experiences, attitudes and/or results of majority groups conceal the nuances between groups. An example relates to domestic violence research and sexuality, where subgroup analyses indicate that bisexual women are much more likely to have experience domestic violence than heterosexual women (Donovan and Barnes 2019). 

We can however examine the results through a sensitivity analysis within a demographic grouping, whatever the amount of people in them, to see if estimates with their confidence intervals are at least comparable to the whole sample without introducing formal testing which would be underpowered. These should be interpreted with caution and seen as hypothesis driving and included in the Statistical Analysis Plan.  If using large datasets such as data from large cohort studies or routine electronic health or social care data, planned sub-groups could be introduced that are sufficiently powered if there is a valid reasoning around using the demographic subgroup to do this.

As of 27 November 2024, the NIHR will require all applicants for domestic programme awards to detail how they will ensure inclusion is considered and built into the whole research lifecycle including how they are collecting demographic data (NIHR 2024). Differences within groups can be starker than differences between groups, however, and intersectionality (Crenshaw 2016) draws our attention to how being, for example, Black African and gay, or Pakistani, older and female, may lead to very different experiences of health and social care compared to studying those identities in isolation (Riggs and das Nair 2012).

With qualitative data analysis, it is important to reflect on your codes and themes as they emerge and consider whether these resonate with the different groups in your sample, or whether some speak more to particular groups than others. Acknowledging negative cases which contradict dominant findings is a key aspect of robust qualitative data analysis and writing up (Patton 1999), but it is also important to interrogate whether there are any patterns in the data that are pertinent to demographic characteristics or specific groups. For example, it might be that most participants spoke positively about their experiences of a new health or social care service, but what about those who were negative or ambivalent? Did they have any demographic characteristics that need to be acknowledged and that might highlight an intervention that is not sufficiently inclusive of that group, or the need for further research about that group’s needs? Particularly with thematic analysis, it is important not to be so focused on specific themes that you become detached from wider contextual factors such as participants’ age, sex, gender, ethnicity or sexual orientation and their related experiences of structural and health inequalities.

Have diverse perspectives been incorporated to support the interpretation of results?

It is also valuable to ensure not only that more than one person is involved in the coding and generation of themes, but also that these researchers occupy different social positions and therefore will interpret the data from a range of vantage points. Practising reflexivity (Probst 2015) is key to recognising how your own identity, lived experience and values might affect your analytical decisions and interpretations.

It is beneficial to involve your public contributors who could bring diversity of lived experience and identities in your data analysis. In some studies, public contributors could conduct some of the analysis; for example, identifying themes and coding qualitative interviews. At the very least, results should be presented to your public involvement group to verify that the narrative around the findings and conclusions drawn are not influenced by the research team’s unconscious bias, but reflect the data being seen. Consideration should be given to how the results are presented to public contributors to ensure they are accessible, and that public contributors are able to use the information to engage in meaningful feedback and discussion.

How will you describe participants’ EDI characteristics?

People often describe other people in relation to themselves, and this can result in people who are often the objects of research being described in ways which may not accurately reflect their identities and experiences. When reporting participants’ demographic characteristics, what language will you use?  Who came up with these names/categories? Did the people you’re describing have a say in what they are called?

Terms to describe people’s identities are constantly evolving, meaning that wordswhich may at one point have been acceptable or analytically convenient may now be considered offensive.  The term ‘BAME’, standing for Black, Asian or Minority Ethnic is increasingly being rejected by those to whom it is applied (Fakim et al. 2020; Inc Arts UK. 2020). This is because of its tendency to clump together people primarily because they are ‘non-white’.  This kind of conflation is sometimes done to gain statistical power by merging existing smaller groups into one big group or to summarise results more concisely.  However, the communities which ‘BAME’ refers to are far from homogeneous and sub-categories of characteristics should not be clumped together where possible.  Government guidance on how to collect, analyse and report ethnicity data is available and may be helpful for you to consider.

Disabled people or people with impairments may subscribe to the social model of disability which describes disabled people as being disabled by an inaccessible and discriminatory society. This is in contrast to the biomedical model of disability which focuses on disability being caused by a person’s impairment (Inclusion London 2022). The biomedical model continues to dominate medicine and medical research, but campaigners are advocating for society – including access to health and social care services and research – to be less disabling.

We should ask ourselves what are we hoping to determine when presenting demographic data  or differences between demographic groups? Demographic data might be used to highlight:

You should check with a diverse group of people with the characteristics being researched to determine how to name categories, whether it be ethnicity, gender, sexuality, disabled people, or people living with a health condition. However, you should also be prepared for challenges in arriving at a consensus: views among populations who share a characteristic can be varied and what may be acceptable to some individuals may be considered offensive by others (e.g. Vincent, 2018 in relation to transgender and non-binary gender identities). Asking participants to self-identify might be the most empowering option, but there are ethical tensions when this data then needs to be coded and aggregated for statistical analysis.

Maintaining anonymity

As well as the considerations above to prevent alienating people from participating in or engaging with research, we also need to take care to protect participants’ anonymity when presenting their demographic data. These risks are amplified when we have low numbers in certain subgroups and when we report participants’ intersecting identities. For example, if we quote from an interview with someone who identifies as a Black, disabled, lesbian woman, this might only describe one or two participants in the sample. Recognising intersectionality is important as different intersections of characteristics have different experiences (Crenshaw 2016, TEDTalk), but it is clear to see that if this data were made public, it could identify people in the data. This risk is heightened with studies of a small geographical area.

With quantitative data, a good rule of thumb is if a table entry has less than five people in it then state ≤5 for the number and not the actual count. For a list of possible identifying variables see (Hrynaszkiewicz et al. 2010). For good practice in sharing data see the UKRI guidance on best practice in the management of research data.  With qualitative data, although detailed ('thick') description adds to the richness of qualitative reporting, careful anonymisation or removal of identifying details may be needed.

Key messages

• It is good practice to consult with groups of people concerning how you will label them in your research. Diverse patient and public involvement is essential to inform these decisions.
• Inclusion is important to build into quantitative and qualitative data analysis. With quantitative data, subgroup analysis should be considered for demographic characteristics where appropriate, but formal statistical testing should be avoided if under-powered to do so.
• When reporting data which includes participants’ demographic data, ensure disclosure control measures are in place to prevent making individual participants identifiable.

• The INCLUDE Ethnicity Framework includes prompts related to quantitative data analysis and reporting for clinical trials in worksheets 3c/3d.
• Inclusion London provides useful guidance about the social model of disability and cultural model of deafness.

Clark, L.T. (2018) ‘Increasing Diversity in Clinical Trials: Overcoming Critical Barriers’, Current Problems in Cardiology, 44(5): 148-172.

Crenshaw, K. (2016) The urgency of intersectionality, TED Talk. Available online at: https://www.youtube.com/watch?v=akOe5-UsQ2o (accessed 12/01/21).

Donovan, C and Barnes, R. (2019) ‘Re-tangling the concept of coercive control: A view from the margins and a response to Walby and Towers’ Journal of Criminology and Criminal Justice, 21: 2, 242-257.

Egede, L.E. (2006) Race, Ethnicity, Culture, and Disparities in Health care, Journal of General Internal Medicine, 21(6): 667-669.

Fakim, N. and Macaulay, C. (2020) “Don’t call me BAME”, BBC News, 30^th June. Available online at: https://www.bbc.co.uk/news/uk-53194376 (accessed 5/05/21).

Hussain-Gambles, M. (2003) ‘Ethnic minority under-representation in clinical trials. Whose responsibility is it anyway?’ Journal of Health Organ Management, 17(2): 138-43.

Hrynaszkiewicz, I. et al. (2010) Preparing raw clinical data for publication: guidance for journal editors, authors, and peer reviewers. BMJ Research Methods & Reporting, 340:c181.

Inc Arts UK: diversity and inclusion in the arts (2020) BAMEover statement. Available online at: URL: https://incarts.uk/%23bameover-the-statement (accessed 5/05/21).

Inclusion London (2022) The social model of disability. Available online at: https://www.inclusionlondon.org.uk/disability-in-london/social-model/the-social-model-of-disability-and-the-cultural-model-of-deafness/ (accessed 20/12/21).

INVOLVE (2012) Strategies for diversity and inclusion in public involvement: Supplement to the briefing notes for researchers. INVOLVE, Eastleigh. Available online at: https://www.invo.org.uk/wp-content/uploads/2012/06/INVOLVEInclusionSupplement1.pdf  (accessed 5/05/21).

NIHR (2020) Strengthening our commitment to equality, diversity, inclusion and patient and public involvement and engagement (PPIE). Available online at: https://www.nihr.ac.uk/documents/strengthening-our-commitment-to-equality-diversity-inclusion-and-patient-and-public-involvement-and-engagement-ppie/24697 (accessed 22/02/22).

Patton, M.Q. (1999) ‘Enhancing the quality and credibility of qualitative analysis’, Health Services Research, 34(5 part 2): 1189–1208.

Probst, B. (2015) ‘The Eye Regards Itself: Benefits and Challenges of Reflexivity in Qualitative Social Work Research’, Social Work Research, 39(1): 37-48.

Tarin, J.J. et al. (2015) ‘Deficiencies in reporting results of lesbians and gays after donor intrauterine insemination and assisted reproductive technology treatments: a review of the first emerging studies’ Reproductive Biology and Endocrinology, 13:52.

UKRI (2015) UKRI Guidance on best practice in the management of research data. Swindon: UKRI. 

Vincent, B.W. (2018) ‘Studying trans: recommendations for ethical recruitment and collaboration with transgender participants in academic research’, Psychology & Sexuality, 9(2): 102-116.

There are many reasons why a diverse research team is a stronger team. Often, people with similar characteristics are drawn to one another. However, idea generation and creativity flourishes when teams include individuals with a mix of life experiences, values and characteristics (Powell 2018; Rock et al. 2016). There are countless examples of what can go wrong when diverse perspectives have not shaped the research. These include devices being designed to suit men’s bodies rather than women’s bodies (Fidler 2020; Hutchison 2019); drugs being less effective for, or having serious side effects on, female patients or patients from particular ethnic groups (Mak et al. 2018; Parekh et al. 2011); and barriers to research participation for people with low socio-economic status or those living in rural areas (Mbuagbaw et al. 2017; Sharrocks et al. 2014). A truly diverse research team is more likely to conceive of, design and conduct research that will meet the needs of a diverse population that includes under-served and under-represented groups.

When researching populations with certain demographic characteristics or who are from an under-served group, it is good practice to ensure that there is a senior researcher or public co-applicant on your team who shares that characteristic. This signals to public contributors and prospective participants that a demographic characteristic which they share is valued and understood. This will strengthen your team’s credibility and the potential for your research to be appropriate and effective. This does not mean that only women can conduct research about women, or only Black people can research Black people’s health. Rather, it questions the acceptability of researching women’s health, or Black people’s health, without a woman or a Black person on your research team. A rallying call from marginalised groups is “nothing about us without us”. Populations with a long history of being oppressed (for example, women; minority ethnic groups; lesbian, gay, bisexual, transgender, queer and intersex (LGBTQI+) people; and disabled people) may be suspicious of researchers and may be more likely to participate in studies where the researcher shares their characteristic(s) (Vincent 2018).

These are sensitive discussions to have when forming a research team, especially because not all demographic characteristics are visible. Researchers may feel uncomfortable about divulging personal information about their demographic characteristics to colleagues or to a funding panel and should not feel obliged to do so. In addition, the under-representation of researchers from minoritised groups means that it will not always be possible for your team to reflect the population(s) studied. This relies on wider work to break down structural barriers to the career development of under-represented researchers and clinicians (Rollock 2019; Kilne 2014).

Belonging to a particular group or community does not necessarily mean that someone will have a nuanced understanding of the perspectives and barriers within that community or are able to build relationships across that community. For example, some LGBTQI+ people know very little about the historic oppression that their predecessors have faced or may have little understanding of the intersecting challenges facing LGBTQI+ people who are disabled, of colour and/or who practise a faith. They may have prejudiced attitudes towards other groups within the LGBTQI+ umbrella, or no knowledge at all (Riggs and das Nair 2012).

The importance of shared identities will vary depending on the research topic and the methodology. But, regardless of how diverse the team appears to be, all research teams need sufficient cultural competence to engage effectively and respectfully with the populations which they study (Vickers et al. 2012). The idea of ‘cultural competence’ needs to be approached carefully. At its worst it can be overly simplistic, reducing groups to stereotypes or checklists of customs, preferences and behaviours which emphasise what makes a particular group different from the ‘norm’ (e.g. the stereotype of gay men being sexually promiscuous). This can become essentialist and even offensive, ignoring the wealth of differences within groups (Riggs and das Nair 2012). ‘Cultural competence’ also implies that there is an end-point to being competent. Because cultures are ever-changing, some have argued that ‘cultural humility’ is a more appropriate term, placing emphasis on being continually open to learning and self-reflection (Greene-Morton and Minkler 2020).

Whilst it is important not to make sweeping generalisations, your proposal will be stronger if you can demonstrate how your understanding of structural oppression, health inequalities, cultural differences, communication needs and key contacts informs your research design. If your study focuses on an under-served population, it will reassure the funding committee if you can demonstrate experience of successfully engaging with and/or recruiting participants from this group.

There are different ways of maximising how well-placed your team is to design research that is relevant, sensitive and accessible to the communities that you seek to include. It may not always be possible to include investigators who share the same protected or other characteristic(s) of your participants and who have appropriate cultural and linguistic competence in relation to the study population. However, you could employ and train one or more community development workers or community researchers who have the necessary skills and expertise to act as a bridge between the research team and the community (Prinjha et al. 2020; Tuffrey-Wijne et al. 2020). In all cases, diverse patient and public involvement and collaboration with relevant community stakeholders is critical (Vickers et al. 2012).

Whichever approach you adopt, what really matters is that this diversity can make a difference. Having a diverse group of people around the table is meaningless and tokenistic unless there is a willingness to listen to different points of view, learn from the lived experiences of others, and be open to your research design decisions being transformed through such dialogue.

Key messages

• Diversity enables teams to flourish through increased creativity, innovation and sensitivity to the needs of different groups. You can strengthen your application by showing how a diversity of perspectives and lived experience has shaped your research design.
• It is important to demonstrate your team’s cultural competence and ability to engage effectively with under-served and under-represented groups.

• The Centre for Ethnic Health Research in Leicester delivers training on cultural competence and community engagement
• Pittsburgh University – video about the importance of cultural humility
• The Centre for Ethnic Health Research has produced a video capturing patient and public perspectives on ethnic diversity in research. This includes public perspectives on the importance of researchers’ cultural competence, awareness of historical context and shared identities with the communities being researched.

Bhambra, G.K. (2017) 'Brexit, Trump and 'methodological whiteness': on the misrecognition of race and class', British Journal of Sociology, 61(S1): S214-S232.

Fidler, H. (2020) ‘PPE: ‘one size fits all’ design is a fallacy that’s putting female health staff at risk’, Nursing Standard. Available online at: https://rcni.com/nursing-standard/opinion/comment/ppe-one-size-fits-all-design-a-fallacy-thats-putting-female-health-staff-risk-160536 (accessed 20/12/21).

Ford, C.L. and Airihenbuwa, C.O (2010) 'Critical Race Theory, Race Equity, and Public Health', American Journal of Public Health, 100(S1): S30-S35).

Hutchison, K. (2019) ‘Gender Bias in Medical Implant Design and Use: A Type of Moral Aggregation Problem?’, Hypatia, 34: 570-591.

Kilne, R. (2014) The “snowy white peaks” of the NHS: a survey of discrimination in governance and leadership and the potential impact on patient care in London and England, London: Middlesex University.

Mak, A.C.Y. et al. (2018) ‘Whole-Genome Sequencing of Pharmacogenetic Drug Response in Racially Diverse Children with Asthma’, American Journal of Respiratory and Critical Care Medicine, 197: 1552-1564.

Mbuagbaw, L. et al. (2017) ‘Considerations and guidance in designing equity-relevant clinical trials’, International Journal for Equity in Health, 16: 93.

Parekh, A. et al. (2011) ‘Adverse effects in women: implications for drug development and regulatory policies’, Expert Reviews in Clinical Pharmacology, 4(4): 453–466.

Powell, K. (2018) ‘These labs are remarkably diverse — here’s why they’re winning at science’, Nature, 558: 19-22.

Prinjha, S. et al. (2020) ‘Including ‘seldom heard’ views in research: opportunities, challenges and recommendations from focus groups with British South Asian people with type 2 diabetes’, BMC Medical Research Methodology, 20: 157.

Riggs, D., & das Nair, R. (2012) ‘Intersecting Identities’ in das Nair, R. and Butler, C. (eds) Intersectionality, Sexuality and Psychological Therapies: Working with Lesbian, Gay and Bisexual Diversity. Oxford: Blackwell Publishing, pp.9-30.

Rock, D., Grant, H. and Grey, J. (2016) ‘Diverse Teams Feel Less Comfortable — and That’s Why They Perform Better’, Harvard Business Review. Available online at: https://hbr.org/2016/09/diverse-teams-feel-less-comfortable-and-thats-why-they-perform-better?ab=at_art_art_1x1 (accessed 20/12/21).

Rollock, N. (2019) Staying Power: The Career Experiences and Strategies of UK Black Female Professors, London: UCU. Available online at: https://www.ucu.org.uk/media/10075/Staying-Power/pdf/UCU_Rollock_February_2019.pdf (accessed 20/12/21).

Sharrocks, K. et al. (2014) ‘The impact of socioeconomic status on access to cancer clinical trials’, British Journal of Cancer, 111: 1684–1687.

Tuffrey-Wijne, I. et al. (2020) ‘Developing a training course to teach research skills to people with learning disabilities: “It gives us a voice. We CAN be researchers!”’, British Journal of Learning Disabilities, 48(4): 301-314.

Vickers, T., Craig, G. and Atkin, K. (2012) Research with black and minority ethnic people using social care services, London: NIHR School for Social Care Research.

Vincent, B.W. (2018) ‘Studying trans: recommendations for ethical recruitment and collaboration with transgender participants in academic research’, Psychology & Sexuality, 9(2): 102-116.

The NIHR Mission is to improve the health and wealth of the nation through 'research'. To achieve this at scale, clear plans need to be in place to share the findings and outputs of the research as widely as possibly, through dissemination. Dissemination is defined as an active approach to spreading of research findings to the target audience via determined channels using planned strategies. Secondly, there is a need to map out the steps to upscaling an intervention or translating knowledge and guidance into practice, through implementation. Impact is the ‘demonstrable contribution that your research makes to society and the economy, of benefit to individuals, organisations and nations’ (NIHR 2022).

Dissemination is needed to communicate the key messages of the research to as wide an audience as possible.  This can include academic papers to an academic audience, but also executive summaries, lay summaries, infographics, videos and blogs. By using varied formats and media and budgeting for translation into different languages, your results can reach further and influence people who have either been disillusioned with research or have had little opportunity to engage with it.

It is particularly important that you explain in your proposal how you describe your dissemination plan, including how you will take the research findings back to the communities who have been involved, as public contributors or participants, and outline the next steps to them. This is key to sustaining relationships and ensuring that under-served communities are not left feeling shut out of the outcomes of the research and that their trust and good will has been exploited. Depending on who has been involved in your research, you may need to consider:

Making plans for implementation, and including questions or a work package about implementation in your study, is key to ensuring that your study has a credible and prompt ‘pathway to benefit’ for patients, service users and/or NHS, public health or social care services.

Implementation studies usually have outcomes which measure both effectiveness of the research/intervention/tool and effectiveness of the implementation strategy (Proctor et al. 2010). Some implementation studies have two primary outcomes; one effectiveness and the other, implementation to reflect this.  Researchers often concentrate on the effectiveness of the intervention and delay considering implementation until much later. 

 If the diversity  of the groups included and appropriate study design for these groups are not factored into the intervention effectiveness phase, the population that the intervention has been tested on could be quite narrow. The researchers may then have limited time, funds and mechanisms for making the intervention more inclusive at the implementation effectiveness stage.  For example, if up until this point the research has only been conducted with English-speaking participants, or only at urban sites with good hospital/university links (Smith et al 2016) or with people under 65 (Bourgeois et al. 2017), then the resultant implementation of the research could be contributing to health and care inequalities and not targeting the highest need. 

Prioritising research inclusion from the outset of any study is essential to avoid missed opportunities to tackle – or even the risk of exacerbating – health and care inequalities. Facilitating diverse and inclusive public involvement, research participation and stakeholder involvement, through a considered research design, will maximise the likelihood of the research findings being successfully mobilised in diverse and complex real-world settings. This is key to achieving equitable impact that benefits those in greatest need, thus helping to reduce health and care inequalities.

Key messages

• Dissemination needs to be tailored to the different and diverse audiences which your findings need to reach.
• Implementation to diverse communities needs to be considered at the design process, and not through adaptations further down the line.
• Adopting inclusive approaches to dissemination and implementation from the outset is more likely to lead to impact which includes reducing health and care inequalities.

• Plain English Campaign – guidance on writing in plain English
• Dos and don’ts on designing for accessibility – blog post by Karwai Pun
• Centre for Ethnic Health Research Equality Impact Assessment Toolkit to assist with planning the impact of research on different demographic groups
• NIHR knowledge mobilisation and impact guidance – for support with planning and useful resource

Bourgeois, F.T. (2017) Exclusion of elderly persons in randomized clinical trials of drugs for ischemic heart disease, Journal of American Geriatrics Society, 65 (11): 2354-2361.

Egli, V. et al. (2019) ‘Disseminating research results to kids: practical tips from the Neighbourhoods for Active Kids study’, Kōtuitui: New Zealand Journal of Social Sciences Online, 14(2): 257-275.

NIHR (2020) ‘Strengthening our commitment to equality, diversity, inclusion and patient and public involvement and engagement (PPIE). Available online at: https://www.nihr.ac.uk/documents/strengthening-our-commitment-to-equality-diversity-inclusion-and-patient-and-public-involvement-and-engagement-ppie/24697 (accessed  20/12/21).

NIHR (2021) Easy-read report: The risk of forced marriage for people with learning disabilities from South Asian communities. Available online at: https://evidence.nihr.ac.uk/alert/easy-read-risk-forced-marriage-people-with-learning-disabilities-south-asian-communities/ (accessed 5/05/21).

Proctor, E. et al. (2010), Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda, Journal of Administration and Policy in Mental Health, 38 (2): 65-76

Smith, T.A. et al. (2016) ‘Selecting, Adapting, and Implementing Evidence-based Interventions in Rural Settings: An Analysis of 79 Community Examples’, Journal of Health Care for the Poor and Underserved, 27(4A):181-193.

Sometimes in research you might want to carry out your study on a very homogeneous sample that will show less variation or ‘noise’ to improve the chance of seeing a difference in your analysis. Although this makes sense numerically, and in the realm of personalised medicine where you tailor a drug to a person’s genotype or phenotype it may be justified (Vogenberg et al. 2010), the purpose of most studies is to improve the health and social care of all groups of people within the population being studied.

All research inclusive of trials and qualitative studies should collect demographic data of the sample studied. This is usually presented in a table, or could be in narrative form in qualitative studies.  This demographic table should include such characteristics as age, sex, gender, and ethnicity. Collecting this data is now an expectation of NIHR-funded research projects. In addition, researchers should evidence how their recruitment strategies will be inclusive. The NIHR INCLUDE pages have a number of questions that can help with making your research more inclusive.    

It is important to consider how well your planned study sample will reflect the population to which the results will be applied and eventually, if the study is successful, implemented on. If the population is heterogenous in terms of demographic characteristics, then your sample should reflect this; otherwise, you could be contributing to health or care inequalities.

Your sample should be relevant to the population and should avoid having unnecessary exclusion criteria. These exclusion criteria sometimes have deep-rooted assumptions; for example:

• not including older people in research on digital interventions due to assumptions about digital literacy
• excluding non-English speakers due to not being aware of or not wanting to use interpreter services
• assuming that people who don’t speak English as their first language wouldn’t have a high level of English comprehension
• assuming that people with a learning disability are not able to consent to take part in research
• not including women and those with a menstrual cycle due to increased time and costs associated with properly accounting for hormonal changes

If exclusions are stated, these need to be justified and supported by compelling evidence. In some disease areas, exclusion based on personal characteristics is required. For example, for asthma diagnosis children less than 5 years old are often excluded due to difficulty in gaining objective tests (NICE 2021). However, if we consider ethnicity as a characteristic, we find that although asthma has a similar prevalence amongst British South Asian people as compared to British white people, South Asian people have a disproportionately higher rate of A&E attendances for asthma attacks (Griffiths et al. 2001). This suggests that they may have different needs or require adjusted logic models when creating an asthma intervention. Without this ethnic diversity included in recruitment, an intervention could fail in this part of the population.  It is important to match your sample to the study population, including the groups of people in greatest need.

In some research it may be justified to use oversampling in order to get an adequate sample of a minority group  in your research and there are several methods to do this (Vickers et al. 2012; Kalton 2009). In other studies research is carried out in geographical areas where disease or care burden is smaller than other areas that would benefit more from such research (Bower et al. 2020). Research is often carried out on samples located near major research centres and more rural areas are neglected (Smith et al. 2016). NIHR is increasingly keen to see that research is being conducted where the need is, and not only in large cities or leading university hospital trusts.

It is important to demonstrate the diversity of your sample through data collection. If this data capture is carried out in feasibility or pilot work, there is still a chance to tailor the study to accommodate a greater diversity of demographic characteristics which would make the outputs and findings of research more generalisable. NIHR acknowledge that diverse sampling might cost more, and from 27 November 2024 require research inclusion costs to be specified to enable inclusive research with diverse participants.

Key messages

• Sample characteristics must be demonstrated using descriptive statistics to demonstrate the diversity and generalisability of your sample.
• If using exclusion criteria, these need to be justified and evidenced and not based on assumptions.

• INCLUDE resources from Trial Forge for designing trials; these include key questions to ask about your planned trial and worksheets
• Draft DAISY guidance from the Wellcome Trust on suggested question formats for collecting survey data about people’s demographic characteristics
• Centre for Ethnic Health Research’s toolkit for increasing the participation of Black, Asian and minority ethnic communities in health and social care research

Bower, P. et al. (2020) ‘Is health research undertaken where the burden of disease is greatest? Observational study of geographical inequalities in recruitment to research in England 2013–2018’, BMC Medicine, 18: 133.

Griffiths C. et al. (2001) ‘Influences on hospital admission for asthma in south Asian and white adults: qualitative interview study’, BMJ, 323(7319): 962-966.

Kalton, G. (2009) ‘Methods for oversampling rare subpopulations in social surveys’, Survey Methodology, 35. No 2, 125-141.

NICE (2021) Asthma: diagnosis, monitoring and chronic asthma management, NICE guideline [NG80] (2017) Last updated: 22 March 2021. Available online at:  https://www.nice.org.uk/guidance/ng80/resources (accessed 5/05/21).

Smith, T.A. et al. (2016) ‘Selecting, Adapting, and Implementing Evidence-based Interventions in Rural Settings: An Analysis of 79 Community Examples’, Journal of Health Care for the Poor and Underserved, 27(4): 181-193.

Vickers, T., Craig, G. and Atkin, K. (2012) Research with black and minority ethnic people using social care services, London: NIHR School for Social Care Research.

Vogenberg, F.R., Barash, C.I and Pursel, M. (2010) ‘Personalized Medicine: Part 1: Evolution and Development into Theranostics’, Pharmacy and Therapeutics, 35(10): 560-576.

Researchers are often unsure about how to budget for research inclusion, and whether adding in additional costs to make studies inclusive will make their bids less competitive. In response to these concerns, NIHR’s latest guidance for applicants emphasises that not only is budgeting for research inclusion welcomed, but, as of 27 November 2024, will be an explicit requirement for all projects.

Reaching out and developing relationships with under-served groups can be time-intensive and costly. However, it is essential in order to conduct research  which meaningfully addresses health and care inequalities and provides equitable opportunities for research participants and public contributors. It is also key to avoiding research waste, as has also been argued by the authors of the INCLUDE Ethnicity Framework.

“Time and money invested in trials recruiting diverse populations are likely to deliver more relevant research, with a consequent reduction in research waste and misallocated healthcare resources in the longer term” (Treweek et al. 2021: 9-10)

The need for research to be inclusive to avoid waste or exacerbating existing health, care and social inequalities relates to all types of research, not just trials. However, there is a lack of guidance on how to accurately cost for the research inclusion aspects of research projects. In the remainder of this section, we outline some of the additional costs and time commitments which inclusive research entails, as well as some steps that can be taken to minimise these costs.

1. What additional costs need to be factored in when designing inclusive research?

2. How might embedding EDI extend the timelines of your research?

There is no one-size-fits-all approach, and each study’s needs should be considered carefully. Any inclusion costs should be clearly justified in applications, just as other design and cost decisions are. The following table lists some of the possible additional costs that may need to be budgeted for when conducting inclusive research. This list is not necessarily exhaustive, and not all these costs will be relevant to every research project.  

3. Are there steps that researchers can take to reduce costs?

The above list of potential costs may seem daunting and it may not be realistic to try and accommodate them all. There are steps that you can take to try to reduce these costs, as follows:

NIHR are committed to funding costs associated with inclusive research. Researchers should include the costs necessary to deliver the research inclusively and to reach and retain a diverse sample of participants, in line with what is required to answer their research questions. As with all costings associated with their research, the inclusion costs should be clearly justified by relevant evidence or design decisions included in the application. Proposals for such costs are assessed on a case-by-case basis within the context of the research application. It is important to offer a strong and compelling justification for the need to increase the engagement and participation of minority and/or under-served groups, and a clear explanation of the costs that this entails.

• NIHR’s finance guidance for applicants (updated November 2024), which includes the types of research inclusion costs that can be included in bids.
• NIHR’s payment guidance for patient and public involvement provides advice on how to plan out the budget at the research design stage. Although it focuses on public involvement, much of it is also applicable to budgeting for participant expenses and inclusion during data collection. 
• The Centre for Ethnic Health Research can quote for support with public involvement, recruitment, conducting interviews and focus groups in multiple languages, translation and cultural adaptation

Treweek, S. et al. (2021) ‘Developing the INCLUDE Ethnicity Framework – a tool to help trialists design trials that better reflect the communities they serve’, Trials, 22: 337.