WHO report on AI interpreting

In an increasingly connected world, simultaneous interpretation is a crucial pillar of international communication. With the growing popularity of artificial intelligence (AI) tools, the World Health Organization (WHO) has conducted a comprehensive study on the reliability of these technologies.

This in-depth research was carried out by the WHO’s Interpretation Team (INT). To evaluate the quality of AI interpretating, the team performed 90 tests using the six official languages of the WHO.

The process involved:

• Selecting an AI interpretation platform.
• Evaluating the interpretation quality for each speech.
• Analyzing the potential reputational risks associated with each translation.

This methodology enabled a direct comparison between the AI’s performance and the quality standards required for official WHO meetings, focusing on the core expertise of the INT team.

The resulting report, which highlights the major weaknesses of AI interpretation, provides a critical perspective on the limitations of these solutions and underscores the irreplaceable importance of human expertise. This article delves into the detailed conclusions of this pivotal study to provide you with a clearer understanding.

The study assigned an average score of only 46% to all AI interpretations. This mediocre performance is due to several key weaknesses:

• Lack of Contextual Understanding: AI struggles to interpret nuances, intonation, emotion, and cultural context. It cannot anticipate speeches or react appropriately to speakers’ hesitations, which directly affects the fidelity and fluidity of the message.

• Issues with Proper Nouns and Cultural References: The translation of names of people, places, or slogans is often incorrect. The report gives the striking example of the phrase “Joy Bangla” (the national slogan of Bangladesh), which was interpreted by the AI as a woman’s name and even mistakenly attributed to a president. Similarly, the name of an African leader was incorrectly gendered.

• Technical Translation Errors: AI has difficulty grasping the meaning of technical terms. For example, the word “transmission” was translated as “transportation,” and hepatitis became “Ebola” in an Arabic translation.

• Literal Translation Problems: Idiomatic expressions, figures of speech, or cultural references are often translated word for word, which makes them incomprehensible or, worse, offensive.

The WHO report identified an alarming number of reputational risks (between 1 and 9 per speech). These risks are particularly serious as they can have a direct impact on the credibility of both speakers and organizations.

• Distorted Speeches: AI can distort statements, render them incoherent, or even make them sound ridiculous. An incorrect interpretation of a country’s or organization’s name can lead to serious diplomatic incidents.

• Gender Bias: The report highlighted instances of gender bias, such as incorrectly assigning a gender to a speaker, which can be perceived as disrespectful.

• Offensive Translation: The literal translation of cultural expressions or slogans can be deemed offensive, exposing both the organization and the speaker to criticism.

The report also highlights other serious issues that were not evaluated in detail during the study:

• Confidentiality and Data Security: The use of AI for interpretation raises significant questions about the security of exchanged data and the control over information.

• Legal Liability: In the event of a translation error with legal or financial consequences, the question of legal liability remains unclear. A human interpreter is bound by a code of ethics, which is not the case for AI.

In conclusion, while AI interpretation may be useful in low-stakes internal contexts, its use for important meetings with external stakeholders is strongly discouraged. The WHO study confirms the irreplaceable value of human interpreters, whose expertise and professional ethics ensure the quality, reliability, and security of multilingual communication.

Read the World Health Organizasiton’s report on IA and interpreting HERE.

On the same topic: « Interprètes humains ou IA » 

About our company, our humans interpreters and our experience: ColinguaRSI