llama.cppCopy wiki link

llama.cpp is an open-source C and C++ inference project for running large language models (LLMs) on a wide range of local and cloud hardware. It is closely associated with the GGUF model format and supports quantized models, command-line use, and an OpenAI-compatible server.

Running llama.cpp locally can keep prompt content on the endpoint, but locality is a deployment property rather than a brand guarantee. Model downloads, plugins, remote clients, telemetry, logs, and exposed server interfaces can reintroduce egress or access risk.

Private AI competitors treats llama.cpp as a foundational runtime for the local tier described in Local AI.