Whisper

OpenAI's Whisper is a groundbreaking open-source automatic speech recognition (ASR) model that has set a new standard for transcription accuracy. Trained on 680,000 hours of multilingual data, it handles transcription, translation, and language identification across 99 languages with impressive robustness. The model comes in multiple sizes (tiny to large-v3), allowing users to balance accuracy against computational requirements. Local deployment is straightforward with a clean Python API, and the community has built numerous wrappers and optimizations like faster-whisper. For those preferring cloud access, OpenAI's API offers transcription at a very competitive $0.006/minute. Strengths include exceptional noise resilience, strong multilingual support, and zero-cost self-hosting. Limitations include higher compute demands for larger models, occasional hallucinations on silent audio segments, and no real-time streaming in the base implementation. As a research tool, the published methodology and open weights make it invaluable for audio ML experimentation. While not technically an LLM, its transformer architecture and open-source nature make it a cornerstone of the open AI ecosystem.