LLM Semantic Caching: Stop Paying Twice
Exact-match vs semantic caching for LLM calls: a small Python cache, LiteLLM and RedisVL setups, and real scores showing why restart and stop nginx collide.
Tag: ai
All the articles with the tag "ai".
My 125B Local Run Ran Out of Weekend
A 125B MoE ran an agent job locally for 72 hours and made real progress. A free 120B cloud route finished the same job in 7. Here is where the time went.
What the B in 12B Actually Means
What the B in 7B, 12B and 125B means, how MoE total vs active parameters differ, and why quantization changes the memory math. A guide for local LLM users.
A Terminal Built for AI Coding Agents
Run Claude Code, Codex CLI, or Aider in parallel with one git worktree per agent, one tmux window each, and a bell that tells you which agent is waiting.
Context Window Math: Paying to Remember
Every chat turn re-sends the whole history. See the token math, what prompt caching saves on the API, and what a long context costs in VRAM when you run local.
Claude Skills vs MCP: What Each One Is For
Agent Skills package know-how, MCP servers package access. Here is how each one loads into context, what each one costs, and when your home lab needs which.
Multi-GPU LLM Inference on Consumer Cards
Two used 3090s or a mismatched pair can run a 27B model. Compare llama.cpp layer split with vLLM tensor parallelism, PCIe limits, and a verdict on which to run.
Blender MCP vs Headless bpy Scripts
Blender MCP is great for building one model with an agent watching screenshots. Headless bpy scripts win for batches, parallel agents, and automated checks.
Dify vs Open WebUI: Chat or App Builder?
Dify and Open WebUI both front your local LLMs, but they solve different problems. Compare footprint, RAG, auth, APIs and licenses, and see how to run both.
Agent Memory Without a Vector DB
Give a local LLM agent long-term memory with Markdown files or SQLite FTS5. Working code, token math, and the point where you finally need embeddings.
Local LLM JSON Output That Doesn't Lie
Constrained decoding makes local LLMs emit valid JSON in llama.cpp, Ollama, and vLLM. It guarantees syntax, not truth. Working curl, Pydantic, and retry code.
Train a Tiny GPT, Part 5: Fine-Tuning
A 5.8M-parameter GPT scores 1.626 bits per byte on Holmes. Gemma 4 E4B scores 0.953 untrained, and a 34-minute QLoRA run on an 8 GB GPU gets it to 0.865.