AI and inclusion: CTOs urgently need to take up the challenge

• AI
• Design
• Digital equality
• User experience

Share on Twitter : AI and inclusion: CTOs urgently need to take up the challenge (New window) Share on Linkedin : AI and inclusion: CTOs urgently need to take up the challenge (New window)

In recent years, there has been a spate of scandals involving the reproduction of all kinds of biases by automated systems and AI models, which can all too easily perpetuate racial and sexual prejudices. First in line to be taken to task over these issues, CTOs (Chief Technical Officers) often pay the price for a lack of diversity in development teams, which contributes to the creation of biased products. “There are few solutions to this problem apart from awareness raising and communication focused on content that promotes inclusive technology to get teams on board,” explains tech-sector inclusion consultant Marcy Charollois, who adds that this context has been made even more problematic by the emergence of a reactionary movement opposed to inclusive technology in the United States. “You even have Elon Musk tweeting that DEI (diversity, equity and inclusion policies) must DIE, because many people in the white majority in the sector also feel oppressed, which has desacralized the need for an inclusive vision in the design of products and services.”

Inclusion doesn’t come from the top down, it’s something that emerges from the bottom up, and rarely involves management, because it upsets economic imperatives.

Moreover, inclusion is not just a matter of gender equality, which is why Marcy Charollois argues that it is not enough to include more women on development teams. “We mustn’t limit this to the idea of having more women if we’re only going to have white women from the same backgrounds. We need a diversity of origins, genders and individuals from the LGBTQIA+ community if our teams are to reflect society as a whole.”

CTOs should take part in more digital responsibility initiatives

“Economic imperatives sometimes take precedence over the need to provide users with equal access to content” points out Nastasia Saby, who works as a machine learning engineer for a media group and is also a member of the Duchess France network of women developers. “For example, advertising is gendered, and the content and advertising recommendation tools developed by media services can reinforce gender stereotypes.” One solution is to encourage CTOs to become more involved in digital responsibility initiatives, although it is not clear how this can be achieved: “In my experience, CTOs rarely talk about digital inclusion, which is mainly promoted by informed and concerned people at the grass roots level. Inclusion doesn’t come from the top down, it’s something that emerges from the bottom up, and rarely involves management, because it upsets economic imperatives,” explains Saby.

For Emilie Sirvent-Hien, the responsible AI program manager at Orange, “there are two elements to take into account: we need to include diversified profiles in the teams and think upstream about the profiles of the users who are going to use these systems.” This step is too often ignored by CTOs who see it as a constraint: “Managers tend to delegate these responsibilities, which they see as burdensome, and to develop solutions, which at the end of the day, are insufficiently effective and may even give rise to scandals,” adds Sirvent-Hien. However, the deployment of AI has obliged CTOs to take on a broader role. They have new responsibilities relating to the design and integration of these projects. Not only do they have to guarantee the integrity of results delivered by these services, but they also need to ensure that the manner in which they are developed complies with AI ethics.

For Nastasia, there is also an underlying issue with motivation and the attractiveness of development jobs that are on offer: “As the designers of digital objects, developers are creative professionals who sometimes find engineering more exciting than results. They also tend to work for companies that take on technologically interesting projects, which are not always so vigilant when it comes to ethics and inclusion.”

Including more people in the design services

According to Emilie Sirvent-Hien, one solution could be to adopt a more reflective approach to responsible design: “We need to set up discussion forums that involve individuals with widely diverse profiles, otherwise developers will never talk to people affected by their output. If, for example, women are to take charge of annotation for an AI project, they need to be able to interact with data scientists to understand the development cycle for the product and how it is supposed to work.” And there is an urgent need for progress given that the AI Act, which came into force on August 1^st, will make companies that deploy AI systems legally accountable, even when they had no involvement in their development. “Companies developing these models are only obliged to document what they’re doing, but at the end of the day, we’re the ones deploying these services to customers and users.”

Reporting in the IT press tends to focus on who will now be responsible for AI risk management, but it is less vocal about strategies to improve inclusion in development teams and digital inclusion in software products. For Marcy Charollois, this is just one aspect of a wider problem: “There is hardly any respect for the Digital Services Act on X, which is the social networking service most favoured CTOs and developers who are culturally averse to  other platforms like LinkedIn. This is just one reason why it is very easy for tech companies to be drawn into a logic of preserving the sta    tus quo, because most of these people are constantly consuming content that is very much aligned with their reality tunnel. We end up with development teams that are similar to the point where they seem to have been cloned, with the same dress codes and the same assimilation process for new recruits.”