Tag: llm

LiteLLM proxies every LLM, local or cloud, behind one OpenAI-compatible endpoint. Pair it with vLLM for GPU-backed serving and ditch the SDK sprawl.

System prompts are your secret weapon. How they work, why they matter more than you think, and 5 patterns that actually change model behavior.

Q4_K_M is the default, but it's not magic. When Q3, Q5, or Q6 makes sense. How to benchmark quantization tradeoffs on your hardware.

Ollama can load one model at a time on limited hardware. How to switch between models, use CPU offloading, and manage VRAM intelligently.

What's the actual difference between context window and token limit? Why one model says 8K and another says 128K. A practical breakdown.

Most people use OpenAI's embeddings because it's easy. But local embeddings exist. How to pick and when it actually matters.

Ollama keeps models in VRAM after every request. Control GPU usage with keep_alive, force-unload via the API, and check memory to stop the reload cycle.

Stop paying per-token to ask questions about your docs. Build a local RAG pipeline with Ollama and ChromaDB: Docker Compose, Python code, chunking strategy.

Connect n8n to Ollama for automations that classify, summarize, and triage instead of just moving data around. Three practical workflows plus prompt design tips.

Text Generation Web UI vs KoboldCpp: setup, model formats, samplers, APIs, and performance compared so you can pick the right local LLM frontend fast.

Compare LangGraph, CrewAI, and AutoGen with real Python examples, local-LLM wiring for each, and a no-nonsense breakdown of which one to actually use.

Self-hosting a ChatGPT alternative? Open WebUI owns local Ollama models; LibreChat handles Claude, GPT, Gemini, and more. Setup, RAG, and trade-offs compared.