이 누리집은 대한민국 공식 전자정부 누리집입니다.

한상넷

비즈니스 정보

Biz News Korea(Eng)

Naver’s HyperCLOVA X outperforms rivals in Korean AI test

Collected: 2024.02.27

Distributed: 2024.02.28

Source: Go Direct

이미지 확대

[Courtesy of Naver Cloud]

Naver Cloud announced on Tuesday that its HyperCLOVA X has achieved higher scores than OpenAI and Google’s generative artificial intelligence (AI) models in the South Korean AI performance evaluation system called KMMLU, Measuring Massive Multitask Language Understanding in Korean.

KMMLU is a performance evaluation metric established by the prominent open-source language model research team HAE-RAE in Korea.

The evaluation consists of 35,030 questions of expert-level knowledge across 45 domains such as humanities, social sciences, and science & technology.

About 80 percent of the questions are related to globally applicable extensive knowledge, including mathematical inference capabilities. The remaining 20 percent assess the model’s ability to solve Korea-specific problems, such as the geography of the Korean Peninsula and domestic laws.

The test questions are composed in Korean, allowing for a more accurate assessment of the AI’s understanding of the language and evaluating its universal capabilities along with local knowledge to provide a comprehensive judgment for Korean users, according to Naver.

[Courtesy of Naver Corp.]

According to the KMMLU research paper, HyperCLOVA X achieved higher scores than OpenAI’s GPT-3.5-Turbo and Google’s Gemini-Pro. It also outperformed OpenAI’s GPT-4 based on Korea-specific knowledge criteria.

Building on the proven performance competitiveness through KMMLU, Naver Cloud plans to further develop HyperCLOVA X into a Sovereign AI solution that has both security and performance.

By Pulse

List

Previous: Shinsegae to initiate group-wide overhaul

Next: Starbucks Korea halts sales of Perrier’s sparkling water