The ERR's Novaator portal has conducted an experiment examining the capabilities of major AI language models in understanding the Estonian language and culture. The study revealed that these models are often unaware of the intricacies and subtleties that characterize the Estonian language, leading to improper responses to culturally significant inquiries. For example, when prompted with a famous line from Estonian literature, the models provided a disjointed response instead of an appropriate interpretation.

In their experiment, Novaator tested the free versions of five popular AI language models by posing culturally and linguistically specific questions about Estonia. They inquired about content from prominent literary works, such as Lennart Meri's 'Hõbevalge,' and even simple linguistic queries like the number of vowels in the word 'jäääär.' Notably, the Grok language model performed the best, followed by Claude Sonnet, Gemini, and ChatGPT, showcasing varying levels of competence among the models.

The findings of this experimentation raise significant concerns regarding data sharing practices involving Estonian-language data with these AI models. Issues of copyright and data protection emerge, emphasizing the necessity for clarity and lawfulness when sharing linguistic and cultural datasets that reflect a nation’s unique identity and heritage. This highlights the need for ongoing discussions around the responsible use of AI technology in preserving language and culture.