Skip to content

Local AI ​

Orrery's AI features run entirely on-device — nothing leaves your machine. There are two interchangeable backends; pick one in Settings → AI & search. When AI is off or unreachable, every AI affordance is hidden rather than broken.

What it powers ​

  • Repo summaries — a synthesized "what is this / what's been happening" blurb on each card, generated on demand or in bulk.
  • Commit messages — drafted from your staged diff in the repo drawer.
  • Changelogs — summarised from recent history.
  • Daily briefing — a one-line "here's where things stand" across your workspace.
  • Resume catch-up — what changed in a repo since you last looked (see The repo drawer).
  • Semantic search — embeddings-backed search over your repos. (Ollama backend only.)

Choosing a backend ​

Ollamallama.cpp (bundled)
SetupInstall Ollama separatelyBundled in the AppImage — download a model
ModelsAny Ollama modelA GGUF you download in Settings
Embeddings / semantic search✅— (generation only)
Best forExisting Ollama users, semantic searchZero-dependency, out-of-the-box generation

The backend selector takes effect immediately, and the panel only shows the controls relevant to the engine you picked.

Setup — Ollama ​

  1. Install Ollama and make sure it's running (default endpoint http://localhost:11434).
  2. Pull a small chat model and an embedding model:
sh
ollama pull qwen3:0.6b
ollama pull nomic-embed-text
  1. In Settings → AI & search, choose the Ollama backend, confirm the endpoint, enable AI, and pick your models. Hit Test to verify the connection end-to-end.

Settings — AI & semantic search

Model choice

A small model like qwen3:0.6b is plenty for the short summaries Orrery generates and keeps things fast. Point the endpoint at a remote Ollama host if you'd rather run the model elsewhere.

Setup — llama.cpp (bundled) ​

The AppImage bundles a local llama-server engine, so there's nothing to install separately. The .deb, .rpm and AUR packages stay lean and don't ship one — install llama.cpp yourself (so llama-server is on your PATH) or use the Ollama backend.

  1. In Settings → AI & search, choose the llama.cpp backend.
  2. Click Download model to fetch a small default GGUF (≈400 MB) into ~/.local/share/orrery/models/. You can also point it at any GGUF URL.
  3. Enable AI and generate a summary — the engine starts on demand and shuts down with the app.

The engine binary is discovered in this order: an explicit path you set in Settings, then ~/.local/share/orrery/bin/, then llama-server on your PATH. A bundled runtime lives at <prefix>/lib/orrery/llama-runtime/, resolved relative to the installed binary (inside the AppImage, usr/lib/orrery/llama-runtime/); on first use Orrery copies it into ~/.local/share/orrery/bin/ so the same lookup finds it, even from the AppImage's read-only mount. If you build from source or install the deb/rpm, supply a llama-server by any of those routes.

Generation only

The llama.cpp backend serves text generation. Embeddings — and therefore semantic search — stay on the Ollama backend, so that one feature is hidden while llama.cpp is selected.

Where data is stored, and clearing it ​

Summaries and embeddings are cached in ~/.local/share/orrery/cache.sqlite. A summary is keyed to the repo's last commit, so it regenerates after new work lands. Use Clear AI cache in settings to drop all summaries and embeddings.

Turning it off ​

Uncheck Enable AI features. The grid, drawer, and toolbar drop every AI control — no empty placeholders, no broken buttons.

Released under the MIT License.