Tag: ai

AMD finally has a fast, open source local LLM server that uses both GPU and NPU. If you've been jealous of Nvidia users, Lemonade is worth your time.

JSON mode forces a model to return valid JSON, which helps extraction and hurts reasoning. When to use it, when it degrades output, and a hybrid approach.

Claude Code found a Linux vulnerability hidden for 23 years. You can use the same AI code auditing approach to find bugs in your own projects before attackers do.

Temperature and top_p control randomness in LLMs. No probability theory needed. Just practical intuition and how to tune them.

Before you download a 70B model, calculate if it fits. The formulas, the gotchas, and a quick calculator you can actually use.

RAG breaks documents into chunks. But what chunk size? Too small and context is lost. Too large and semantic search fails. Here's how to pick.

System prompts are your secret weapon. How they work, why they matter more than you think, and 5 patterns that actually change model behavior.

Q4_K_M is the default, but it's not magic. When Q3, Q5, or Q6 makes sense. How to benchmark quantization tradeoffs on your hardware.

Piper vs Coqui TTS compared: speed, voice quality, Docker setup, and Home Assistant integration. Run offline neural TTS on your own hardware, no cloud fees.

What's the actual difference between context window and token limit? Why one model says 8K and another says 128K. A practical breakdown.

Most people use OpenAI's embeddings because it's easy. But local embeddings exist. How to pick and when it actually matters.

Ollama holds models in VRAM after every request. Set keep_alive, force an unload through the API, and cap how many models stay resident at once.