Detailed content
🧰 Preparing the groundwork
A local AI lab doesn’t need anything exotic — it needs three things in order: an updated operating system, free disk space, and a terminal at hand. This is the foundation everything in the next tracks will run on. Skipping this part is the number one cause of “I installed it and it didn’t work.”
🆕 New here?
"Driver" is the program that lets the system communicate with a piece of hardware — in our case, the graphics card (GPU). Without the right driver, the GPU is in the machine, but the AI software can’t use it. "Terminal" is the text window where you type commands (Terminal on macOS, PowerShell on Windows, any shell on Linux) — almost all setup happens there.
It's worth having the mental model of stack that makes local AI possible. Each layer builds on the one below: the model runs only if the runtime exists; the runtime uses the GPU only if the driver is installed; and all of it runs on the hardware.
Read the stack from bottom to top: the hardware supports the driver, which frees up the GPU for the runtime, which finally loads the model. This module covers layers 1 and 2—the rest comes in Track 3.
✓ Ready ground
- ✓Operating system in the current version
- ✓30–50 GB of free disk space for the models
- ✓GPU driver installed and detecting the card
- ✓Terminal open, and you’re comfortable pasting commands
✗ Makeshift setup
- ✗OS stuck on an old version without updates
- ✗Disk nearly full — model doesn’t even finish downloading
- ✗GPU present, but no driver (runs everything on the CPU, slowly)
- ✗Fear of the terminal — gets stuck on the first command
🍎 macOS (Apple Silicon)
If you have a Mac with an M-series chip (M1, M2, M3, M4), good news: the foundation is already almost ready. The GPU is integrated, and Apple exposes it via Metal — there's no separate driver to install. There's just one piece left: a package manager for installing command-line tools. That's Homebrew.
🆕 New here?
- Metal = Apple’s graphics API; it’s what runtimes like Ollama use to run the model on a Mac’s GPU. It comes with the system.
- MLX = Apple’s machine learning framework, built for Apple Silicon’s unified memory (more in topic 5).
- Homebrew = the most popular package manager for Mac; installs terminal programs with a single command.
⌨️ Practical example: install Homebrew
Objective: to get your Mac ready to install Ollama, Python, and whatever else Track 3 requires. Paste this into Terminal and follow the on-screen instructions.
How to check: close and reopen Terminal, then run brew --version. You should see something like Homebrew 4.x. If the command isn’t found, the installer prints two lines at the end eval ... — paste them in to add the brew to PATH.
💡 Practical tip
On a Mac, you no it messes with the GPU driver — Metal is already there. The advantage of Apple Silicon is the unified memory: RAM and VRAM are the same pool, so a Mac with 32 GB+ can run models that would stall on a GPU with little VRAM (revisit Track 2.1). The tradeoff is lower bandwidth: it runs large models, but not as fast.
🐧 Linux + NVIDIA (CUDA)
The most common pairing in AI workstations is Linux + NVIDIA GPU. This is where the part most people forget comes in: the NVIDIA driver and the CUDA. Without them, your R$ 10 mil graphics card sits idle and the model runs slowly on the CPU.
🆕 New here?
"CUDA" is NVIDIA's software layer that lets programs use the GPU to calculation — not just for graphics. It’s exactly what makes AI run fast on the GPU. CUDA comes with/depends on the NVIDIA driver, so installing the right driver takes care of most of it.
⌨️ Practical example: can the machine see the GPU?
Objective: confirm that the driver is active and see how much VRAM you have — the number that determines the model size. Paste this into the terminal.
How to check: nvidia-smi should show a table with the GPU name, driver version, and memory (e.g., 24576MiB). If you get command not found, the driver still isn’t installed—follow the steps below using what ubuntu-drivers devices mark as recommended.
Installing the driver on Ubuntu is a short sequence:
Update the system
Run sudo apt update && sudo apt upgrade to start from a clean base before installing the driver.
Find the right driver
ubuntu-drivers devices lists compatible drivers and marks the recommended. Trust it the first time.
Install
sudo ubuntu-drivers autoinstall installs the recommended version. To pin a version: sudo apt install nvidia-driver-550.
Restart and check
Restart and run nvidia-smi again. Now the table with the GPU and VRAM should appear. Done: CUDA is available.
🪟 Windows (+ optional WSL2)
Windows runs local AI very well, especially on PCs with gaming NVIDIA GPUs. The simplest route: install the driver GeForce (or Studio) normally and use Ollama or LM Studio directly on Windows. If you want a Linux environment without switching machines, there’s WSL2.
🆕 New here?
"WSL2" (Windows Subsystem for Linux 2) is a Linux for real running inside Windows, with access to the GPU. You get the Linux command ecosystem — many AI tutorials assume Linux — without needing dual boot or another machine.
🪟 Native Windows
- •Simpler: install the GeForce driver and the app (Ollama/LM Studio)
- •Great if you just want to run and chat with the model
- •Fewer steps, fewer things to break
🐧 With WSL2
- •Linux environment for following tutorials and scripts exactly
- •Good for Docker, Open WebUI, and dev workflows (Tracks 3 and 4)
- •The NVIDIA GPU appears inside WSL with the current Windows driver
⌨️ Practical example: enable WSL2
Objective: have Ubuntu inside Windows. Open PowerShell as administrator.
How to check: restart, open the app Ubuntu from the Start menu and create a username/password. If you have an NVIDIA GPU with an up-to-date driver on Windows, nvidia-smi run inside Ubuntu's should already detect the card — you don't need to install a driver inside WSL.
🔀 AMD (ROCm) and Apple (MLX) — the overview
Not everyone has NVIDIA—and that's okay. It's worth knowing the landscape of CUDA alternatives, because it determines what works today, hassle-free, and what still takes patience.
🆕 New here?
- ROCm = AMD’s answer to CUDA: the layer that lets AMD GPUs run AI computations. Newer and less universally supported than CUDA, but evolving quickly.
- MLX = Apple’s framework for Apple Silicon; specialized libraries (e.g., mlx-lm) take advantage of the chip's unified memory.
NVIDIA · CUDA
Mature. The most common and best-supported approach.
Linux or Windows. Works on practically all local AI hardware.
AMD · ROCm
Evolving. Works, but check your card's compatibility first.
Linux is the most solid ground. Ollama already runs on several AMD GPUs.
Apple · Metal/MLX
Native. On a Mac with an M chip, it just works.
No driver. MLX for those who want to go beyond Ollama on Mac.
✓ The path with the least friction today
- ✓NVIDIA + CUDA on Linux or Windows
- ✓Apple Silicon + Metal/MLX on Mac
- ✓Ollama abstracts all three—you switch machines and the command stays the same
✗ Where it still hurts
- ✗ROCm requires checking whether your AMD GPU is on the supported list
- ✗Very new or very old GPUs may still lack support
- ✗Mixing manufacturers in the same machine makes setup more complicated
💡 Practical tip
Don’t get stuck on this choice. If you already have a machine, use what it has — Ollama (Track 3) detects the GPU automatically in most cases. If you’re buy for local AI today, NVIDIA on PC or Apple Silicon on Mac are the paths with the fewest surprises.
🧹 Lab cleanup
The foundation is in place, but here’s the part no one tells you: keeping the lab organized and healthy. Three habits prevent most future headaches — organize where the models are stored, version your configs, and monitor temperature/power consumption.
⚠️ Alert: heat and power
Running a large model pushes the GPU and CPU to 100% for several minutes at a time—that heats up and consumes real power. On a laptop, the impact is greater: the machine may spin up its fans, throttle (reduce performance to protect itself), and drain the battery quickly.
- •Don’t run it on a bed/sofa that blocks the air vents; use a hard, well-ventilated surface.
- •Before letting an agent 24/7 (Track 4), check the temperature under load and your electricity bill.
- •If the machine shuts itself down under load, it’s almost always overheating—clean out the dust and improve ventilation.
✓ Good hygiene
- ✓A dedicated folder for the models (you know where everything is)
- ✓Configs and Modelfiles versioned in git
- ✓Keep an eye on nvidia-smi / Activity Monitor under load
- ✓Close what you're not using before loading a large model
✗ Carelessness
- ✗Models scattered across Downloads, Desktop, and half a dozen folders
- ✗Configs only in your head—you format the drive and lose everything
- ✗Ignore the temperature until the machine freezes
- ✗Dozens of things competing for the GPU at the same time
⌨️ Practical example: environment summary
Objective: a quick snapshot of your system — OS, RAM, and GPU — so you know what you're working with before choosing models.
How to check: save as resumo-do-ambiente.sh and run bash resumo-do-ambiente.sh. You should see three blocks — System, RAM e GPU. Save this output: it's the starting point for deciding on the model size (Track 2.2) and opening a ticket if something goes wrong.
Self-check (optional): what is the CUDA and why does it matter in a Linux + NVIDIA lab?
📌 Module summary
Next module:
2.4 — Private network with Tailscale