Tag: llm

LangChain does everything and LlamaIndex does one thing brilliantly. Here's how to pick the right RAG framework without regretting it at 2 AM.

RAG is the default answer for giving LLMs access to documents. But chunking, embedding, and retrieval introduce failure modes that a virtual filesystem sidesteps entirely.

Google's Gemma 4 is the best open model they've shipped yet. Here's how to pull it, run it, and actually use it for real work with Ollama on your own hardware.

1-bit models store weights as -1, 0, or 1. That sounds insane until you see them run a 100B parameter model on a laptop CPU. Here's what's actually happening.

AMD finally has a fast, open source local LLM server that uses both GPU and NPU. If you've been jealous of Nvidia users, Lemonade is worth your time.

JSON mode forces a model to return valid JSON, which helps extraction and hurts reasoning. When to use it, when it degrades output, and a hybrid approach.

Temperature and top_p control randomness in LLMs. No probability theory needed. Just practical intuition and how to tune them.

Before you download a 70B model, calculate if it fits. The formulas, the gotchas, and a quick calculator you can actually use.

vLLM, llama.cpp, and Ollama all run local LLMs, compare throughput, memory use, GPU support, and which fits your hardware.

RAG breaks documents into chunks. But what chunk size? Too small and context is lost. Too large and semantic search fails. Here's how to pick.

LiteLLM proxies every LLM, local or cloud, behind one OpenAI-compatible endpoint. Pair it with vLLM for GPU-backed serving and ditch the SDK sprawl.

System prompts are your secret weapon. How they work, why they matter more than you think, and 5 patterns that actually change model behavior.