Large language models (LLMs) are rapidly becoming integrated into our daily lives, influencing everything from the information we consume to the decisions we make. But how well do these models, and the underlying reward models that guide their behavior, perform outside of English? A groundbreaking new study introduces M-RewardBench, the first large-scale benchmark for evaluating reward models across 23 languages. Researchers discovered a significant performance gap between English and non-English languages, revealing that the digital world's promise of seamless translation and cross-cultural understanding might be further away than we think. The study explored reward models' performance across various tasks, including chat, safety, reasoning, and translation, with surprising results. While some models, particularly generative models, exhibited stronger multilingual capabilities, others struggled significantly, especially with less-resourced languages. This performance disparity raises critical questions about fairness and equitable access to AI technologies. Imagine an AI assistant that gives excellent advice in English but falls short when asked the same question in Swahili or Vietnamese. This inconsistency highlights the urgent need to train and evaluate reward models on diverse datasets that reflect the richness and complexity of global languages. The study also uncovered a surprising link between translation quality and reward model performance. Higher-quality translations led to improved performance across all model types, emphasizing the interconnectedness of different aspects of AI development. Looking ahead, M-RewardBench offers a crucial tool for researchers to develop more robust and inclusive reward models. As AI continues to evolve, this research emphasizes the importance of ensuring these powerful technologies serve everyone, regardless of their language.