Luce Megakernel: CUDA Fusion Beats Apple Silicon Efficiency
· 6 min read
A single CUDA kernel for all 24 layers of Qwen 3.5-0.8B delivers 1.87 tok/J on an RTX 3090, matching Apple's M5 Max at 2x the throughput.
local-ai
Local-ai Articles
10 articles in this category
Local AI in the Wild: What Real Users Are Actually Running
· 12 min read
54 comments from developers running Gemma 4, Qwen 3.5, and other local models — the hardware, the benchmarks, the frustrations, and the wins.
Gemma 4 for Local OCR: Self-Hosted Document Processing with Ollama and TurboQuant
· 8 min read
How to use Gemma 4 as a local OCR engine — processing images and PDFs through Ollama with vision models, no cloud APIs needed. Covers the architecture, TurboQuant's impact on long-context document processing, and a practical Python implementation.
Z-Image-Turbo and Flux 2 Klein 4B: Local Image Generation on AMD iGPU and CPU with stable-diffusion.cpp
· 16 min read
Two of the newest distilled diffusion models — Z-Image-Turbo and Flux 2 Klein 4B — both run locally on AMD integrated graphics and CPU using stable-diffusion.cpp. No NVIDIA GPU required. We benchmark both on a Ryzen 5 PRO 4650U and show how they share the same text encoder to save disk space.
RotorQuant and IsoQuant: Fixing Turbo Quant's Prefill Bottleneck with Clifford Algebra
· 7 min read
How RotorQuant replaces Turbo Quant's expensive 128x128 matrix rotation with Clifford algebra rotors — 44x fewer parameters, 10-19x faster on CUDA, matching attention fidelity on real models.
Google Turbo Quant: Theory, Dense vs MoE Context, and llama.cpp Benchmarks
· 7 min read
A deep dive into Google's Turbo Quant KV cache compression — from the theory of 3-bit compression vs 4-bit, through dense vs MoE context scaling experiments, to a full llama.cpp benchmark with FP16, Q4, and Turbo Quant head-to-head.
llama.cpp: Running LLMs on AMD Vega iGPU with Vulkan
· 9 min read
Getting llama.cpp to work on an AMD Ryzen 5 PRO 4650U with integrated Vega graphics — no NVIDIA, no CUDA, no ROCm. Just Mesa RADV and the Vulkan backend.
FastSD CPU: Run Stable Diffusion on Any CPU — No GPU Required
· 7 min read
Generate images locally using Stable Diffusion on nothing but your CPU. FastSD CPU uses Latent Consistency Models and OpenVINO to produce 512x512 images in under a second on a modern processor — no $5,000 GPU needed.
Local AI Hardware Guide: Why VRAM Matters More Than GPU Speed
· 4 min read
A practical guide to building local AI systems focused on VRAM—the key bottleneck for running AI models locally at usable speeds.
Run Data Center AI Accelerators in Your Workstation: A DIY Guide
· 8 min read
How to repurpose a Tesla V100 SXM2 AI accelerator from a DGX server into your home workstation for running local LLMs at a fraction of GPU costs.