• From autonomous driving to social robotics, this method helps machines master cultural nuances and complex gestures.
• Promising more human-like interaction, this approach remains costly in terms of computing power and requires highly accurate data to anticipate our true intentions.
An AI system that works in France may not necessarily be as successful in another country due to the cultural specificities of users. Based on this observation, researchers at the University of Washington hypothesized that AI can learn cultural values by observing human behavior. In short, AI could absorb values in the same way that children do during their learning process. To conduct their experiment, the researchers used agents whose learning method, inverse reinforcement learning (IRL), is very different from traditional learning systems. “Our results provide a proof of concept demonstrating that AI agents can be endowed with the ability to learn culturally typical behaviors and values directly from observing human behavior,” they explain. This is because this method is based on behavior observation and a continuous reward system.
IRL is useful in robotics for teaching robots to imitate expert movements, but also in understanding and modeling behavior in order, for example, to help psychologists or economists understand individuals’ motivations.
How IRL enables AI to create its own rules
In traditional AI systems, reinforcement learning is used. In order for AI to learn a task correctly, it is given a reward function, such as a points system. “AI seeks to maximize future rewards,” explains Lina Rojas-Barahona, a researcher at Orange specializing in conversational systems. “In IRL, we don’t know the reward function. In some cases, we can’t define reward functions. For example, if we want to teach an autonomous vehicle to drive, we know that there is no single way to drive.” In this case, the idea behind IRL is to collect information about people who know how to drive and learn by observing them. “The AI will then create its own reward function based on its observations and use it to train itself.”
In addition to subtle learning tasks such as driving, IRL is useful in robotics for teaching robots to imitate the movements of experts, but also in understanding and modeling behavior in order, for example, to help psychologists or economists understand individuals’ motivations. In the field of healthcare, the idea is to deduce preferred care protocols by observing the decisions of the best practitioners. In finance, IRL can be used to model the strategies of expert investors to detect market anomalies or automate portfolio management.
A costly method in pursuit of precision
However, this approach has limitations and is costly: it requires recording hours of human behavior, ensuring that the data is representative and qualitative, and processing this data requires significant computing power. Like RL, AI in IRL operates in a repetition loop, but unlike RL, it must learn the reward, compare its results with those of humans, adjust its operation, and redefine the reward again, and so on. “It must predict this reward, and in terms of computing power and latency, this can be problematic since learning and inference are more costly,” explains Lina Rojas-Barahona.
Beyond the material cost, IRL also faces a methodological limitation: the inability of certain systems to anticipate long-term intentions. In an article published in Nature, a team of Chinese researchers developed a method to help social robots navigate smoothly and safely among humans. Current research on robot navigation uses IRL to mimic human trajectories, but it is based on people’s direction rather than their goals, i.e., their destination. Over long distances, robots’ decisions become less accurate. The researchers have therefore developed an approach that explicitly integrates the navigation goal into the robot’s learning process. The robot first trains in a random virtual environment, then refines its behavior by observing humans to understand social norms. This method allows robots to make decisions step by step without having to predict complex future trajectories.
This text has been translated by an artificial intelligence.
Read more :
Lina Maria Rojas-Barahona, Christophe Cerisara. Bayesian Inverse Reinforcement Learning for Modeling Conversational Agents in a Virtual Environment.. Conference on Intelligent Text Processing and Computational Linguistics, Alexander Gelbukh, Apr 2014, Kathmandu, Nepal.
Lina Rojas-Barahona
