LiveBench is a benchmark for large language models designed to be challenging and contamination-free. It was created by researchers primarily affiliated with Abacus.AI and NYU, with contributors from Nvidia, the University of Maryland, USC, and Columbia; lead authors include Colin White, with senior authors including Yann LeCun and Micah Goldblum. Released in June 2024, it was published as an ICLR 2025 Spotlight paper.
LiveBench addresses two common problems in LLM evaluation: test-set contamination and LLM-judge bias. Questions are sourced from recently released material, such as math competitions, arXiv papers, news articles, and datasets, so models cannot have memorized answers during training. Every question has a verifiable, objective ground-truth answer that is scored automatically without using an LLM as a judge.
The benchmark covers 18 tasks across six categories: reasoning, math, coding, language, data analysis, and instruction following. It is updated monthly, with roughly one-sixth of questions replaced per update so the benchmark fully refreshes about every six months. The public leaderboard evaluates all major frontier models from OpenAI, Anthropic, Google, Meta, DeepSeek, Mistral, and others.
LiveBench is free to use and fully open source, with evaluation code on GitHub and datasets on Hugging Face.