How to improve DEI metrics: 6 challenges to overcome

Written by Kimberly Silva, Senior R&D Consultant

The Harvard Business Review Analytic Services (HBR-AS) and the Society for Human Resource Management (SHRM) recently published a study on workplace diversity, equity, and inclusion (DEI) with sobering results. Despite making DEI initiatives a high strategic priority, only 33% of leaders say they’ve greatly improved their workplace. Even fewer employees (29%) think their organization is where it should be with respect to DEI. The results beg the question, why are organizations struggling so much?

Seven experts weighed in on the issue in a panel discussion dedicated to workplace DEI (Silva, K., et al., 2022). They agreed that challenges are partly due to DEI metrics. Before discussing metrics, I need to clearly define (and uncouple) diversity, equity, and inclusion:

• Diversity is the presence of individual differences, like race, gender, disability, sexual orientation, and age.
• Equity is fair treatment and access to resources for all people, regardless of individual differences.
• Inclusion is the degree that all people feel welcomed, heard, and safe to fully participate in groups or other processes.

Now, I can highlight six key metric challenges that limit progress on DEI initiatives.

1. Metrics are not universal.

Metrics for one organization are not necessarily the most salient metrics for another organization. By targeting the wrong metrics, leaders fail to uncover issues that are specific to their organization. According to Felicity Menzies, CEO of a diversity and inclusion consultancy, the legal, cultural, historical, and political nature of each nation, region, and organization should determine which DEI issues to target. Organizations must carefully consider all local challenges before deciding what metrics to collect.

2. Research participation is low.

Many employees do not complete DEI questionnaires or surveys because they distrust the process. They may not trust leaders to make meaningful changes, take their concerns seriously, safeguard their anonymity, or protect them from negative repercussions (Hurt & Dye, 2020). Organizations cannot draw deep insight from incomplete data. So, if faced with low participation, a good starting point is to evaluate confidence in DEI efforts and data management.

3. Sentiment is difficult to capture.

Organizations typically rely on scaled items, open-ended questions, or focus groups to evaluate attitudes or feelings on DEI. While useful, all these methods are notorious for measurement issues, like rating inflation/deflation or socially desirable responding (Sparkman et al., 2018). That’s no surprise; many employees either believe themselves to be or want to present themselves as advocates of DEI. Or, they are hesitant to share negative feedback. To help combat this issue, organizations should test innovative ways to measure DEI that do not suffer from the same metric shortcomings.

4. Data are difficult to interpret.

Major decisions about DEI initiatives depend on high-quality data interpretation. The problem is findings look different depending on how much data are available or how thinly we slice the data. Take race, for example; diversity numbers may seem ‘good’ when racial groups are pooled together (e.g., majority versus minority groups), but unacceptable when specific ethnicities are considered. Or there may be a balanced number of men and women in leadership positions, but not Black women. Often, demographic groups are pooled together because of small numbers, which are difficult to interpret. However, pooling demographic groups discounts within-group differences and impedes DEI change (Wullert, K, et al., 2019). Organizations need to carefully consider how they will interpret data and make plans for handling small and complex datasets before presenting results.

5. Success is hard to define.

What are ‘good’ DEI results anyway? How do we know if an organization is diverse, equitable, or inclusive enough? It’s not always straightforward whether a goal was met. More often than not, organizations only partly reach their goal (e.g., increased gender diversity for new hires but decreased gender diversity among managers). And, because people interpret results differently, the outcome of the findings differ depending on who summarizes the evidence. This challenge has no clear solution. However, organizations would likely benefit from having a diverse panel of individuals to define, evaluate, and regularly revisit markers of success.

6. Results are difficult to communicate.

Communicating results, especially negative ones, can be challenging and risky. It opens the organization up to scrutiny, reveals shortcomings, and implies responsibility to make changes. Organizations may be apprehensive to share results, or they may avoid certain metrics altogether, because they are wary of communicating sensitive information (Leiderman, 2018). However, ignoring or withholding DEI results presents its own consequences. Today’s employees expect their leadership to be transparent. Leaders should therefore discuss communication strategies and plan for difficult conversations, regardless of content and outcomes.

There are many challenges that prevent organizations from successfully improving workplace DEI. However, by addressing metric issues, organizations can remove some barriers and hopefully create a more productive, safe, and happy workforce.

Resources

Harvard Business Review Analytic Services & Society for Human Resource Management. (2021). Creating a culture of diversity, equity, and inclusion: Real progress requires sustained commitment. Harvard Business School Publishing. https://www.shrm.org/hr-today/trends-and-forecasting/research-and-surveys/pages/dei-metrics-.aspx

Hurt, K., & Dye, D. (2020). Courageous cultures: How to build teams of micro-Innovators, problem solvers, and customer advocates. HarperCollins Leadership.

Leiderman, S.V. (2018). How to communicate diversity and inclusion when you aren’t quite there yet. Sustainable Brands. https://sustainablebrands.com/read/marketing-and-comms/how-to-communicate-diversity-and-inclusion-when-you-aren-t-quite-there-yet

Menzies, F. (2018). Meaningful metrics for diversity and inclusion. Include-Empower.com. https://cultureplusconsulting.com/2018/10/16/meaning-metrics-for-diversity-and-inclusion/

Silva, K., Kazmi, M.A., Kirkland, J., Polk, K., Tringale, A., Markell-Goldstein, H., & Oki, T. (2022). Measuring, Interpreting, and Communicating DEI Metrics [Panel]. Society for Industrial and Organizational Psychology Annual Conference, Seattle, WA, United States.

Sparkman, D. J., Eidelman, S., Dueweke, A. R., Marin, M. S., & Dominguez, B. (2019). Open to diversity: Openness to experience predicts beliefs in multiculturalism and colorblindness through perspective taking. Journal of Individual Differences, 40(1), 1–12. https://doi.org/10.1027/1614-0001/a000270

Wullert, K., Gilmartin, S., & Simard, C. (2019). The mistake companies make when they use data to plan diversity efforts. Harvard Business Review. https://hbr.org/2019/04/the-mistake-companies-make-when-they-use-data-to-plan-diversity-efforts