Install OwnLLM

You need a Mac (Apple Silicon or Intel) or a Linux computer (Intel/AMD or ARM) with at least 2 GB of memory and a few GB of free disk space. No other software is required and you never need administrator rights.

The easy way (about 10 MB, then one model)

curl -fsSL https://github.com/ownllm-org/ownllm/releases/latest/download/install.sh | sh

The installer downloads a small package, checks its fingerprint, installs it for your user only, and offers a model that fits your computer. Say no if you would rather choose later.

Prefer to look first? Download install.sh (and SHA256SUMS), read it - it is short - then run sh install.sh. Add --dry-run to see what it would do without changing anything.

Fully offline (no internet on the target computer)

On a computer that is online, download from the release page the package for your system (ownllm-<version>-<system>.tar.xz) and SHA256SUMS, and from the project's Hugging Face page (https://huggingface.co/ownllm/ownllm-models) the model file(s) you want, for example qwen2.5-1.5b-instruct-q4_k_m.gguf for "small". Copy them over, then:

shasum -a 256 -c SHA256SUMS --ignore-missing      # use sha256sum on Linux
tar -xJf ownllm-*.tar.xz && cd ownllm-*/ && sh install.sh --no-model
ownllm get small --from /folder/with/the/model    # checks the file's fingerprint, then installs it

Windows

Native Windows is not supported. Use WSL2 (Ubuntu) and follow the Linux steps.

First steps

ownllm chat              # talk to it
ownllm ask "What is a mortgage?"
ownllm start             # private web page at http://127.0.0.1:8765  (ownllm key shows the key)
ownllm doctor            # health and safety check

Choosing a model

ownllm models lists them. Bigger answers better but needs more memory: tiny (2 GB), small (4 GB), medium (8 GB), standard (12 GB). ownllm get standard adds one (the files come from the project's Hugging Face page). Every file is checked by SHA-256 before it is used.

Remove it

ownllm uninstall (add --purge-models to delete the models too).