Local LLMs concept

How to run an LLM locally

Reference

How to run an LLM locally: choose a model and a compatible app, download the model, load it, and try a small prompt. A local run means inference happens on your device, but an app may also offer cloud or remote models. Check which model and location are selected before sharing sensitive text, and use the test to confirm that the setup works for your task.

Reviewed 2026-10-09

What a local run means

A large language model (LLM) is a model that generates text from an input. Running one locally means its inference—the process that produces a response—happens on your device. In LM Studio documentation checked 2026-10-09, the app distinguishes local models from cloud models and models running on another device. A familiar chat window alone does not tell you where inference happens, so check the selected model and its location before you begin.

The workflow, in order

First, choose a local-model app that supports your device and the model format you plan to use. Next, read the model card: check its intended uses, limitations, evaluation information, and license. Choose a model and variant that the app supports and that fits your device. Download it through the app or the maker’s official process, then select the downloaded model and choose the local option if the app offers multiple locations. Start with a short, ordinary prompt so you can see whether it loads and responds.

Common snags and the data path

A model may fail to load if its format or variant is unsupported, or if it does not fit the available resources. Longer context—the amount of text the model can consider at once—can affect memory needs, so a resource estimate may depend on context length. If loading fails, check the app’s compatibility information and try a smaller or supported variant. Local is not synonymous with private: inspect the data path for cloud models inside local apps, remote models, and any logs or telemetry your setup uses. Downloading a model can also require an internet connection.

Check that it works as intended

Ask a simple test question and confirm that the app identifies the selected model as local. Check whether the reply is responsive enough for your task and whether the app reports a load or compatibility problem. If you need to check offline operation, first download the model and required runtime, then disconnect from the network and repeat the test. A reply by itself does not establish where inference ran; the selected model and location, together with an offline test, provide a more useful check.

Questions people ask

How do I run AI on my computer?

Use an app that supports local inference, download a compatible model, select the local option, and try a short prompt. Check the selected model and location in the app.

How do I install a local LLM?

Install a local-model app using its official instructions, then choose and download a model the app supports. Check the model card for intended uses, limitations, evaluation information, and license.

Can a local LLM work without internet?

Some apps can run a downloaded local model offline, but finding and downloading models or runtimes may need a connection. Test offline only after the needed files are on your device.

Does running an LLM locally mean my data stays on my computer?

Not automatically. Check whether the selected model is local, cloud, or remote, and review the app’s data path, including any logs or telemetry your setup uses.

Learn more

Download Local Models (LM Studio Bionic documentation, checked 2026-10-09)
Shows how to find, download, and select a local model in this app.
Model Cards
Explains the model-card details to check before choosing a model.
Offline Operation (LM Studio documentation, checked 2026-10-09)
Describes which local-app tasks can work offline and which use network requests.

What changed

Sources

Part of Local LLMs: Choose a model that fits Run a small local test Verify the data path and troubleshoot