Tag: ollama

vLLM, llama.cpp, and Ollama all run local LLMs, compare throughput, memory use, GPU support, and which fits your hardware.

Ollama can load one model at a time on limited hardware. How to switch between models, use CPU offloading, and manage VRAM intelligently.

Most people use OpenAI's embeddings because it's easy. But local embeddings exist. How to pick and when it actually matters.

Ollama holds models in VRAM after every request. Set keep_alive, force an unload through the API, and cap how many models stay resident at once.

Stop paying per-token to ask questions about your docs. Build a local RAG pipeline with Ollama and ChromaDB: Docker Compose, Python code, chunking strategy.

Connect n8n to Ollama for automations that classify, summarize, and triage instead of just moving data around. Three practical workflows plus prompt design tips.

Master Ollama with Modelfiles, GPU tuning, API usage, and performance tricks. Stop running 70B models on 8GB VRAM and wondering why everything is slow.

Temperature, top-p, top-k, context length, LLM inference parameters explained so you stop guessing why the model gives weird output.

GGUF, GGML, AWQ, GPTQ, LLM file formats and quantization levels explained: trade-offs between model quality, size, and inference speed.

LLaMA, Mistral, Falcon, GPT, the LLM landscape is crowded. Compare model families, sizes, licensing, and what each is actually good for.

Ollama makes local LLMs easy to start. The only decision that matters is whether the model fits your VRAM. Compose file, real sizes, and the gotchas.