Intersectionality in AI: Challenges and Solutions for Multidimensional Equity

• AI
• Algorithm
• Digital equality
• Trust Security and Transactions

Share on Twitter : Intersectionality in AI: Challenges and Solutions for Multidimensional Equity (New window) Share on Linkedin : Intersectionality in AI: Challenges and Solutions for Multidimensional Equity (New window)

Managing Bias: The Only Solution for Greater Equity.

In terms of equity, we speak of bias in AI when systems perpetuate—or even reinforce—existing prejudices and inequalities between individuals or groups, whether they are privileged or not. If these biases are not controlled, they can lead to discrimination, which is prohibited by law, and undermine public trust in the company providing the service. Given the growing integration of AI into critical areas such as employment, healthcare, the justice system, and credit, it has become particularly urgent to combat these biases. The European Union has enacted the AI Act, a pioneering regulatory framework that requires companies to identify, reduce, and manage biases throughout the AI lifecycle. This includes selecting representative data, designing models capable of detecting and mitigating biases, and continuously monitoring systems.

Effective bias management promotes fairness and performance and fosters transparency and trust, while improving the accuracy of decisions. It helps strengthen organizations’ reputations, ensures regulatory compliance, and attracts ethically minded employees and customers.

Intersectionality reveals the complexity of discrimination in AI. Innovative tools are needed to measure and mitigate these multidimensional biases.

The definition of fairness is highly contextual

Fairness is an ethical issue and therefore inherently subjective. Its definition is highly context-dependent. To measure the biases embedded in an AI model and then potentially mitigate them, it is necessary to go through multiple steps, each of which examines the cultural and operational context in which the model is used.

First, we must examine the datasets used to train the model and its various variables. These datasets contain the information on which the model will base its predictions. Among these, certain variables (e.g., gender, age, ethnicity) may prove to be discriminatory in the context under consideration. It is important to identify these so-called sensitive attributes. They will then be treated differently: they are used to define so-called demographic groups (or subgroups in the case of intersectionality), with respect to which we will seek to maintain equity.

Next, it is necessary to define what would constitute harmful treatment by the model toward an individual or a group and, by contrast, what constitutes an acceptable situation—in the absence of an ideal state of equity.

Once these definitions are established, we can quantify biases and measure the equity of the final model using equity metrics. In the event of bias, mitigation measures must be implemented using various existing tools.

Existing metrics do not allow for the management of intersectionality

Existing equity metrics are quite limited. In fact, most allow for comparing individuals or groups only within a binary framework—that is, by considering only a single sensitive attribute (e.g., gender OR age OR origin, etc.) that can take only two values (e.g., male/female, junior/senior, urban/rural, master’s degree or higher/no master’s degree or higher, etc.).

The reality of discrimination, however, is often more complex and requires what is known as an “intersectional” approach. This approach involves studying how certain subgroups—defined by a combination of several sensitive characteristics (for example, a group of young women living in rural areas)—may experience heightened discrimination. Indeed, these combined situations can lead to forms of discrimination that are more severe than the simple sum of the discrimination associated with each criterion taken separately.

Take, for example, a resume-screening model. Traditional metrics can verify that it performs equally well for men and women. They can also verify that it performs equally well for people with a master’s degree as for those without one. However, they cannot simultaneously guarantee fairness for the four subgroups resulting from the combination of these two criteria: men with a master’s degree, women with a master’s degree, men without a master’s degree, and women without a master’s degree.

A Legal Requirement

The European AI Regulation (AI Act) and various other laws require organizations to ensure the absence of discriminatory bias. European law lists fourteen grounds for discrimination. French law adds nine additional criteria, including sexual orientation, physical appearance, status as a whistleblower, vulnerability resulting from economic circumstances, and so on. By adding the issue of intersectionality to this list of criteria, one must potentially consider all the subcategories resulting from combinations of these different criteria, thereby multiplying the number of population groups to be considered. The grounds for discrimination to be taken into account and the division of the population into demographic groups depend on the context. For example, an individual’s biological sex is a well-known ground for di ; however, in the case of a health model, it is likely that it is appropriate for the model to treat men and women differently, as male and female bodies do not always require the same treatments.

“The challenge, then, is to combat multiple discrimination and, possibly, to measure the intersectional consequences of algorithmic biases. With this multidimensional (or intersectional) approach, it becomes clear that striking a perfect balance among so many statistical thresholds—even for a single requirement—is a complex but essential task.” (source)

Multiple groups, multiple disparities: what technical choices should be made to measure these differences in treatment?

Managing equity through an intersectional lens means ensuring that a model performs equally well for each subgroup, based on a criterion carefully chosen in light of the intended definition of equity. This requires deciding how to compare the model’s performance across different subgroups.  Indeed, while comparing a model’s performance between two groups is straightforward (one can calculate the difference or the ratio), the same cannot be said for more than two groups.

For example, to verify that a recommendation system is as relevant for men as it is for women, it suffices to compare the model’s performance between these two groups, either by difference or by ratio. On the other hand, if we wish to evaluate the relevance of recommendations for four intersectional subgroups (young women, older women, young men, older men), we must then compare four performance metrics simultaneously.

How will we compare the subgroups? Will we compare each pair of subgroups two by two, or each subgroup against the average? Or, instead of comparing subgroup by subgroup, will we directly measure the relationship between the model’s output and sensitive attributes (gender, age, etc.)? All of these methods exist and do not necessarily highlight the same distribution of advantages and disadvantages across subgroups. Each therefore has its own advantages and disadvantages. We must choose the one that best aligns with what we consider fair and acceptable.

Some research studies have proposed equity metrics tailored to intersectionality but have not always clearly explained the reasons for choosing one comparison method over another. This lack of justification creates, at the very least, a lack of transparency that obscures the issue. The lack of justification could also indicate a lack of in-depth reflection on this point. Yet it is important to be transparent and to understand the implications of technical choices, in order to ensure that these implications are normatively aligned with the intended objectives and that the AI model can be considered accountable for the results it produces.

A meta-metric for fairness is needed to reduce complexity

Whether it’s to develop evaluation standards, compare multiple models to choose the most equitable one, or manage bias when training a model, it’s helpful to have a summary score. However, in the case of intersectionality, the proliferation of subgroups creates numerous indicators. These can be difficult for the human mind to grasp. It is therefore necessary to reduce this complexity by enabling the comparison of models based on a single score. This score, called a “meta-metric,” is then applied to the results of each subgroup. This simplifies the comparison and allows for an assessment of a model’s fairness as a whole.

The choice of the meta-metric influences the ranking of models according to the fairness criterion, and potentially the choice of the model ultimately selected.

The meta-metric, which aims to combine all the results obtained for each subgroup into a single overall score, can in fact be calculated in various ways. For example, it can take the form of a standard that counts the number of times a given threshold is exceeded for disparities between subgroups. Another type of meta-metric may choose to measure the information shared between sensitive attributes and the equity metric. These multiple possibilities can be further expanded through variations, particularly via weighting. Greater or lesser weight can be assigned to a group or a disparity based on: the size of the groups, their identity, the discrimination experienced, and so on. Ultimately, several types of meta-metrics are possible, each providing a different interpretation of the results.

As shown in the image below, these different options do not lead to equivalent evaluations. Depending on the method chosen, the model considered the most equitable may not be the same, or at least, the relative rankings of these models’ equity may vary. Even though all methods agree on the trivial case of perfect fairness (Case I in the example below), they may yield different results in more realistic situations where disparities exist. Figure 1 concretely illustrates that the choice of method influences the conclusion: it embodies a normative stance and can reverse the ranking of the models.

Linking the Technical Choice to Practical Implications

To choose among the different aggregation methods, one must understand how they account for the distribution of inequalities across subgroups. The ideal of absolute equity corresponds to a situation where no disparities are observed in the equity measures for each subgroup (see distribution I in Figure 1). This situation is, however, rare and remains more of a goal than an attainable reality. In practice, disparities in inequality will always exist.

The real questions are: How do we want to distribute these disparities? If they persist, do we want to protect the most disadvantaged subgroup by limiting the maximum gap? Are we seeking to reduce systemic biases by limiting the frequency of disparities? Do we accept significant gaps between subgroups, provided they are rare? Or do we want to ensure that the model does not rely on sensitive attributes to make its predictions?

Depending on the prioritized objectives, we must choose an aggregation method that reflects these priorities. This allows us to obtain a measure of intersectional bias that is consistent with—or “aligned” to—the design and values we wish to prioritize.

Outlook

Intersectionality reveals the complexity of discrimination in AI. Innovative tools are needed to measure and mitigate these multidimensional biases. The key is to move beyond the limitations of current tools, which are too narrow relative to the range of possibilities and lack insight into their normative implications and the justifications for the technical choices that led to these implications.

The specific approach to bias assessment within an intersectional framework highlighted in this article is a first step in this direction.

The next step will be to provide the various tools corresponding to the multiple possibilities presented, all of which are necessary to ensure reliable bias management that takes into account the complexities of the world.

Orange Research, through doctoral thesis work, is working toward this goal, and the latest advances in our research on the subject were presented at the FAccT 2026 conference. Feel free to read the article to learn more.

This text has been translated by an artificial intelligence.