Ollama

Ollama is an open-source tool that enables users to run large language models locally on their own computers without requiring cloud connectivity, API keys, or external services. Launched in 2023, Ollama simplifies the process of downloading, configuring, and running LLMs by packaging model weights, configurations, and runtime dependencies into a single manageable system. It supports macOS, Windows, and Linux, with a straightforward command-line interface where users can get started with a single command such as 'ollama run llama3' to download and interact with a model. Ollama provides access to a growing library of pre-configured open-source models including LLaMA, Mistral, Mixtral, Gemma, Phi, CodeLlama, DeepSeek, Qwen, and many others, available in various sizes and quantization levels to match different hardware capabilities. Users can run models on consumer-grade hardware including Apple Silicon Macs, PCs with NVIDIA GPUs, and even CPU-only systems, with models automatically optimized for the available hardware. A key advantage of Ollama is that all processing happens locally, meaning no data leaves the user's machine, making it suitable for privacy-sensitive use cases and air-gapped environments. Ollama exposes a local REST API compatible with the OpenAI chat completions format, enabling integration with existing tools and applications. This has led to a rich ecosystem of third-party integrations including web UIs, IDE plugins, desktop applications, and development tools that use Ollama as a local model backend. Users can also create custom models through Modelfiles that define base models, system prompts, parameters, and adapters, enabling personalized configurations. Ollama supports features like concurrent model loading, GPU acceleration, and multi-modal models for vision tasks. The tool is entirely free and open-source under the MIT license.