What is GGUF?

A model file with weights and metadata

GGUF is a binary format for storing models for inference with GGML and executors based on it. GGML is the software library named in the format documentation. GGUF is designed for fast loading and saving, and for ease of reading. A GGUF file can contain tensors—the multidimensional numeric arrays holding model values—along with metadata. The specification describes tensor names, dimensions, data types and offsets, as well as metadata stored as key-value pairs. This information gives compatible software details it can use when interpreting a model file.

What the file contains

A GGUF file starts with a marker and a format version, followed by counts for its tensors and metadata entries. Metadata uses key-value pairs. Tensor descriptions include a name, dimensions, data type and offset to the tensor data. The format also defines metadata for a model’s architecture and tokenizer, and it can store a chat template: a description of the input format expected by a model. Compatible software can use these details to load the stored data and prepare model inputs.

GGUF supports quantized tensor types. Quantization represents tensor values in a lower-bit form, and some such types can reduce the memory needed for model weights. The format can describe a tensor’s data type, but the file extension alone does not tell you whether a particular program supports that model or quantization. The GGUF specification says quantization-version metadata is not required when a model has no quantized tensors.

A format in the model-loading path

GGUF sits between stored model weights and software that can load and run them. The llama.cpp documentation checked 2026-10-09 says llama.cpp requires models in GGUF format; support in other software depends on that software’s own documentation. A compatible file still needs software that supports its architecture and tensor types. Check those details, along with tokenizer and chat-template compatibility, in the model and runtime documentation.

GGUF and Safetensors are both formats for storing model data, but they serve different workflows. GGUF stores tensors with metadata for inference in GGML-based software. Safetensors is a format for storing tensors safely and quickly. A file’s format alone does not determine the data path or where inference runs: an application may use a local model, a model on another device, or a cloud model.