🧭 Fundamentals
Before installing anything, understand the landscape: why local AI has become a sovereignty movement, what it really is, why “dumb and slow” is less and less true, and how an LLM works under the hood. This is the foundation that makes everything else make sense.
Trail map
Detailed content
🌍 The turning point: why now
The case for "why": restricted access, rising hardware costs, and sovereignty as a way forward.
Frontier models reach selected groups first, while labs around the world accelerate.
Understanding the race explains why relying only on the cloud is risky.
Frontier models, restricted access, open models.
Memory and GPUs under pressure from prices and future demand (robots, drones, cars).
The urgency is to get started while hardware is still obtainable.
Scarcity, a window of opportunity, starting with what you have.
The intelligence runs on your machine, under your control, and no one can shut it down.
Sovereignty is the thread running through the entire course.
Sovereignty, independence, your own foundation.
The three pillars: privacy, unlimited use, and non-removability.
Each pillar enables uses that are impossible in the cloud.
Private, no extra cost, always on.
The three beliefs that hold people back — and why they don't hold up in practice.
Removing the mental barrier is the first practical step.
Good enough, hardware cycle, get started for free.
How the tracks connect, from the why to 24/7 agents.
Knowing the way keeps you oriented.
Prerequisites, progression, destination.
🏠 What local AI really is
Where the model runs, what an LLM is, and the three pillars in practice.
In the cloud, the prompt travels to a data center; locally, everything happens on your GPU/CPU.
It's the difference that explains privacy and cost.
Inference, data center, offline.
A weights file + an inference program that generates text.
Demystifies the “model” and prepares you for the anatomy (1.4).
Weights, inference, tokens.
Prompts and responses aren't sent to third-party servers.
Critical for sensitive data.
Sensitive data, no external logs.
No per-token cost, enabling continuous use and batch processing.
Cost-effective foundation for 24/7 agents.
No rate limits, low marginal cost.
Independence from price, policy, and region.
Resilience is part of the value.
Resilience, personal responsibility.
Cases where cloud or hybrid makes more sense.
Decide based on criteria, not dogma.
Hybrid, peak intelligence, cost-effective.
🐘 Is local dumber/slower? The elephant in the room
The honest objection — and why the gap is closing fast.
Today, many local models are simpler/slower than the best cloud models.
Facing the truth prevents frustration.
Capability gap, realistic expectations.
Open models (GLM, Qwen, Llama, DeepSeek) are getting close to the top.
For most tasks, it’s enough.
Open models, convergence.
The trade-off triangle; local changes the economics.
Consciously choosing the balance point.
Trade-offs, usage savings.
More efficient models make old hardware productive again.
Dispels the fear of “obsolete in 2 years.”
Efficiency, quantization, longevity.
How to read benchmarks skeptically (contamination, cherry-picking).
Decide based on your use case, not the rankings.
Benchmark, contamination, your own test.
What runs great locally vs. what still needs the top model.
Know the right tool for each task.
Chat, embeddings, classification, complex code.
🧬 Anatomy of a local LLM
Weights, tokens, context, quantization, memory, and licenses — no code.
Billions of parameters; more ≈ more capable and heavier.
It's the first number you look at when choosing a model.
Parameters, scale, disk size.
Token = piece of a word; context = how much fits in “working memory.”
Long context uses memory—it affects what fits.
Token, context, KV cache.
Reduce the precision of the weights (FP16→Q4) to fit in less memory.
It's what makes affordable hardware possible.
Accuracy, the quality × size trade-off.
Estimate VRAM/RAM by size × quantization.
Connects directly to the hardware choice (Track 2).
VRAM, RAM, context overhead.
The main open model families and what each range is for.
Know where to start when choosing.
Llama, Qwen, Gemma, Mistral, DeepSeek, Phi.
Difference between Apache/MIT and licenses with usage restrictions.
Avoids problems in commercial use.
License, commercial use, checklist.