Enkokilish (እንቆቅልሽ) are Ethiopian/Amharic riddles. Riddles in Amharic are difficult, require in-depth understanding of the language and are often used to test one's knowledge and reasoning skills. So based on Enkokilish, we've made Enkokilish Bench to evaluate the ability of Large Language Models (LLMs) to understand, reason, and solve Amharic riddles.

This benchmark is built using Evalite as the evals framework, AI-SDK to make calls to Vercel AI Gateway to a variety of models. This benchmark is completely free and open-source from dataset to eval code, and to this visualization site. To quickly setup, clone the repo, set your AI Gateway API Key in the file and just then open and explore. You can run the evals in node mode (just by running ) which enables you to export the results in JSON format, or run the evals in a CI/CD pipeline.