Tagarmharcanna
European languages
Cáilíocht samhlacha ar thagarmharcanna aistrithe teangacha Eorpacha. Tagann na sraitheanna EU21 ón meastóireacht OpenGPT-X a foilsíodh le Teuken 7B Instruct v0.6, a aistríonn ARC, HellaSwag, TruthfulQA agus MMLU go dtí na 21 teanga AE a chumhdaíonn sé agus a thuairiscíonn meán amháin in aghaidh na samhla. Is é an tsraith MMMLU figiúr MMLU ilteangach Google féin do Gemma 4.
The EU21 numbers are OpenGPT-X’s, taken from the Teuken 7B Instruct v0.6 model card and the accompanying evaluation preprint; they are machine-translated benchmarks, not native-language test sets. The MMMLU number is Google’s own. LLM EU has run no evaluation and has not verified either harness.
| Múnlaí | score | source |
|---|---|---|
| Teuken 7B Instruct v0.6 | 0.57 | link |
| Llama 3.1 8B Instruct | 0.563 | link |
| Mistral 7B Instruct v0.3 | 0.527 | link |
| Aya 23 8B | 0.485 | link |
| Pharia-1-LLM-7B-control-aligned | 0.417 | link |
| Gemma 4 31B IT | 88.4 | link |