Detailed content
🛰️ Where the model runs: cloud vs. local
The difference between cloud AI and local AI comes down to one question: where does the account run? In the cloud, your text leaves your machine, travels over the internet, and is processed in another company's data center. Locally, the same setup runs on your own computer—and nothing needs to leave it.
Read the diagram in two lanes. Above (red), the path of cloud: your text crosses the internet to a server that isn’t yours, then comes back. Below (green), the path local: prompt → your GPU/CPU → response, all inside the dashed box—it works even without internet.
🔑 The central idea
Change the where and you change everything that follows from it:
- •Local = no data transit: the prompt never crosses the internet, so there’s no copy on someone else’s server.
- •Local = no external owner: no one between you and the model to charge you, log activity, or block you.
- •Local = works offline: if the internet goes down, the intelligence keeps working.
📦 What an LLM is, concretely
Before you “run it locally,” it’s worth demystifying what this thing that runs actually is. An LLM isn’t magic or a brain: in practice, it’s a large file of numbers + a program that reads text and predicts the next piece. Knowing this takes away the fear and explains why it does (or doesn’t) fit on your machine.
🆕 New here?
"LLM" (Large Language Model) is the AI model that runs behind ChatGPT and similar tools. It consists of two parts:
- Weights: a giant file of numbers (the “parameters”) that store what the model learned. That's what you download.
- Inference: the act of use the weights used to generate a response—the "calculation" your GPU/CPU performs for each word. (Training is something else, much more expensive; you only do inference.)
- Token: the piece of text the model manipulates — usually ~a syllable or part of a word. The model reads tokens and predicts the next one, one at a time.
At its core, generating text is repeating a single step, token by token:
Reads what already exists
The model receives your prompt broken into tokens and “looks” at all of them at once.
Predicts the next token
Using the weights, it calculates which token is most likely to come next. This is inference — the part that puts a load on your machine.
Repeat until finished
It appends the generated token and starts again. Token after token, the response takes shape. That's why the text "starts typing" on the screen.
✂️ How text becomes tokens
The sentence below, split into tokens (each | is an approximate boundary):
Rule of thumb: ~0.75 word per token in Portuguese. Remember this—“context window” and memory cost are measured in tokens (the topic of module 1.4).
🔒 Real privacy
The first of the three benefits of running locally is the most concrete: your prompts and responses never leave the machine. There’s no copy on a third-party server, no conversation log you don’t control, and no data-use policy beyond your own. For some jobs, this stops being a convenience and becomes a requirement.
✓ Local
- ✓The text stays on your machine’s disk and in its memory
- ✓No conversation logs on someone else's server
- ✓You decide what to keep and what to delete
- ✓Nothing is used to train another company's model
✗ Cloud only
- ✗The text is transmitted and processed by third parties
- ✗Logs and retention follow the policy of them
- ✗Data can be used for training, depending on the terms
- ✗Provider leak exposes what you sent
⚖️ When it really matters
- Legal: contracts, parts, customer data protected by professional confidentiality.
- Health: medical records, test results, any patient data.
- Personal and financial: diary, plaintext passwords, bank statements, trade secrets.
- Proprietary code: your company's data shouldn't be sent to an endpoint you don't control.
💡 Practical tip
Privacy isn’t "all or nothing." The healthy default: sensitive data runs locally, something routine can go to the cloud. The point of running locally is for you to power choose — instead of sending everything outside due to a lack of alternatives.
♾️ Unlimited: no toll per prompt
The second benefit changes your economics. In the cloud, you pay per token — every call has a meter running. Locally, after the hardware, the marginal cost of one more prompt is basically electricity. This may seem like a small detail, but it unlocks an entirely new way to use AI.
✓ Local (fixed cost)
- ✓Running all day doesn’t change the per-call bill
- ✓Processing 10,000 items in batches is feasible
- ✓Experiment without worrying about "burning credits"
- ✓24/7 agents become possible (Track 4)
✗ Cloud (pay per use)
- ✗Each token has a cost to send and receive
- ✗Continuous use can lead to costs that scale badly
- ✗You censor yourself to avoid spending money
- ✗Rate limits cut the batch off midway
⌨️ Try it: a batch you would think twice about running in the cloud
Objective: summarize 50 lines of a file, one by one, without looking at any meter. (Works after Track 3, with Ollama installed.)
How to check: it processes the 50 lines in sequence. The only “bill” that goes up is your electricity bill—no API credits were used. This is exactly the pattern that powers the agents in Track 4.
💡 Practical tip
"Unlimited" means within your hardware’s limits: one machine runs a model at a certain speed. You don't pay per token, but you pay for time and energy. The advantage is that the cost is predictable and doesn't scale with each call.
🛡️ It can’t be taken from you
The third benefit is the most underestimated: the model on your disk doesn't disappear. Once downloaded, it doesn’t depend on an active subscription, the provider continuing to exist, your region being supported, or a price that could double tomorrow. The intelligence becomes an asset you own.
✓ It’s yours; it stays
- ✓No subscription required to cut off access
- ✓No country or region blocking
- ✓No price or policy changes catching you off guard
- ✓Works even if the provider disappears from the map
✗ Borrowed; it can go away
- ✗Suspended account = no access, no warning
- ✗The model is "retired," and you lose access to it
- ✗Prices rise and the budget can’t keep up
- ✗Regional restriction keeps you out
🤝 The honest side: with power comes responsibility
Sovereignty isn’t just about gains. When the intelligence is yours, maintenance is too:
- Uptime is yours: if your machine shuts down, the "service" goes down—there’s no support to call.
- Security is yours: keeping the lab secure (network, access) is your responsibility — the topic of Track 2.
- The update is yours: downloading better models when they come out is up to you.
🧭 When NOT to use local
Honesty wraps up the module. Local is powerful, but it isn't the only answer to everything. There are cases where the cloud still wins — and recognizing them is what separates mature sovereignty from dogma. Use this simple workflow to decide case by case:
The workflow isn't "local all the time." It's: sensitive data or heavy use → local; peak intelligence, no hardware or occasional use → cloud (or a hybrid). Sovereignty means keeping the base local and using the cloud by choice.
🚩 Signs that the cloud is the right choice
- Absolute peak: the task requires the smartest model available today, and nothing less will do.
- Without hardware: you still don't have a machine powerful enough for the model size you need.
- Rare use: an occasional question doesn't justify building and maintaining a lab.
📝 Try it: the "would I send this outside?" test
Objective: feel, in practice, where local processing is a requirement for you. Grab a piece of paper (or a file) and list what you no would paste into a cloud chat today. Start with these questions:
- ›Which work documents contain customer data or confidential information?
- ›What code from your company shouldn’t leave for an external endpoint?
- ›What personal information (health, finances, diary) would you like to keep just for yourself?
How to check: everything that made it onto this list is, in practice, your task queue for local AI. Save it — it becomes your concrete “why” throughout the course.
Self-check (optional): what's the essential difference that defines "local AI"?
📌 Module summary
Next module:
1.3 — Is local dumber/slower? The elephant in the room