b0x207/local-ai
1
0
Fork 0
Code Activity Actions
Declarative Local AI
25 commits 1 branch 0 tags 120 KiB
  • Nix 99.1%
  • Shell 0.9%
Find a file
Exact
Ben Landon cb19dc52b7
 feat(qwen3.8): switch to a GSQ-RCO 
This will require testing as it constitutes a change from IQ4 down to
IQ3.
2026-09-28 21:09:28 -07:00
models feat(qwen3.8): switch to a GSQ-RCO 2026-09-28 21:09:28 -07:00
.envrc Initial Commit 2026-07-22 18:13:32 -07:00
.gitignore feat: simplify model definitions with mkModel 2026-07-24 00:39:42 -07:00
flake.lock flake: update inputs 2026-08-17 17:35:46 -07:00
flake.nix feat: dynamic model switching 2026-08-17 17:32:25 -07:00
lib.nix feat: dynamic model switching 2026-08-17 17:32:25 -07:00
README.md feat: simplify model definitions with mkModel 2026-07-24 00:39:42 -07:00
run.sh feat: dynamic model switching 2026-08-17 17:32:25 -07:00
runner.nix feat(qwen3.8): switch to a GSQ-RCO 2026-09-28 21:09:28 -07:00

README.md

Nix For Local AI

Run models in a declarative manner. Get a full toolchain through the power of devshells.

Usage

You can enter the devshell with either nix develop or nix-direnv.

Running the models can be done via the run.sh script.

TODO:

  • Swappable models
  • Make mmap an option for running
  • Configurable context length that can still be constrained by individual models