🛠️ Hardware & Lab
Time to prepare the ground. Learn about AI hardware categories, find out what your machine can handle, prepare your operating system and drivers, and connect everything to a private network with Tailscale so you can use your models from anywhere.
Trail map
Detailed content
🧩 Hardware categories
The AI hardware map: unified memory, VRAM, and bandwidth — and which category your machine fits into.
Unified memory (Apple) combines RAM and VRAM in a single pool — it fits a huge model, but the lower bandwidth makes it slow. Dedicated VRAM (NVIDIA) has less memory, but much higher bandwidth and greater speed.
This trade-off determines everything: model size versus speed.
Unified memory, VRAM, bandwidth.
Mac Studio/Mac with lots of unified memory (e.g., 512 GB) can run models over 200 GB, but slowly.
Great for huge models; not ideal when you need speed.
Apple Silicon, unified memory, large models.
NVIDIA DGX Spark (~128 GB) and AMD Ryzen AI Max "Strix Halo": moderate memory and bandwidth.
Good balance between size and speed; the OS is usually Linux.
DGX Spark, Strix Halo, balance.
RTX 5090 (~32 GB, very fast) and RTX 6000 Pro (~96 GB, expensive). Fast, as long as the model fits in VRAM.
It's the choice when speed is the priority.
RTX 5090, RTX 6000 Pro, dedicated VRAM.
Mac mini, laptops, and PCs under $4k can run small models (Gemma, Phi, small Qwen models), embeddings, and classification.
It still delivers a lot of value—and it's where almost everyone starts.
Small models, embeddings, low cost.
macOS (usability), Linux (workstations), and Windows (gaming GPUs), each with pros and cons.
The OS affects drivers, runtimes, and day-to-day friction.
macOS, Linux, Windows.
🎯 How to choose: what fits your machine
From use case to memory calculation: find the largest model your machine can handle — and how fast.
Start with the use case (chat, code, RAG), not the hardware.
The use case determines the model range — and, from there, the required hardware.
Use case, model range, requirements.
GB ≈ (parameters in B) × (bytes per quantized weight) + context overhead. Table: 7B/13B/34B/70B × Q4/Q8/FP16.
Estimating before downloading avoids the frustrating "it didn't fit".
Parameters, bytes per weight, overhead.
Each hardware profile unlocks a different model range.
Knowing your ceiling keeps you from buying or downloading the wrong thing.
VRAM, RAM, unified memory.
tokens/s = how quickly the text comes out; memory bandwidth dominates here.
Sets what’s comfortable for chat versus batch processing.
tokens/s, latency, bandwidth.
A laptop + Ollama + a 3-8B Q4 model can run today, for free.
Removes the excuse, “I need to buy something before I can get started.”
Zero cost, small model, Q4.
Prioritize memory; use a dedicated GPU when speed matters; add multiple machines later (Tailscale, module 2.4).
Spending in the right order gets you much more for each dollar.
Priority, dedicated GPU, multi-machine.
🧰 Operating system, drivers, and foundation
Get the groundwork ready: OS, drivers, and the layer that lets the GPU run AI — on macOS, Linux, and Windows.
Every lab needs an up-to-date OS, disk space, and a terminal close at hand.
A solid foundation prevents 90% of the problems that come later.
Updates, disk space, terminal.
Apple Silicon uses Metal/MLX, with no extra driver; just install Homebrew.
It's the shortest path to getting a lab running on a Mac.
Metal, MLX, Homebrew.
CUDA is the layer that lets an NVIDIA GPU run AI; install the driver and check with nvidia-smi.
It's the most common setup for workstations and servers.
CUDA, NVIDIA driver, nvidia-smi.
GeForce driver + the WSL2 option for a Linux environment inside Windows.
Makes use of your gaming GPU without changing operating systems.
GeForce, WSL2, Linux environment.
The alternatives to CUDA and what actually works right now.
To realistically assess whether your AMD GPU is suitable for the lab.
ROCm, MLX, alternatives.
Organize model folders, version configs, and monitor temperature/power.
A clean lab scales; a messy lab leaves you stuck at the worst moment.
Organization, versioning, monitoring.
🔗 Private network with Tailscale
Securely access your models from anywhere: a private network that connects your machines as if they were on the same LAN.
Your machines are at home while you’re away — and opening ports to the internet is risky.
That's exactly the pain point Tailscale solves.
Remote access, without opening ports or taking on risk.
Point-to-point mesh VPN over WireGuard; creates a private network among your devices as if they were on the same LAN.
It's the piece that connects the whole lab in one place.
Mesh VPN, WireGuard, tailnet.
Install via script (curl ... | sh) and log in with sudo tailscale up; confirm with tailscale ip -4.
The setup is quick and identical on every machine.
Installation, login, Tailscale IP.
Expose Ollama on the tailnet (OLLAMA_HOST=0.0.0.0) and call it from another device using the 100.x IP.
This is how another machine starts using your model.
OLLAMA_HOST, 100.x IP, tailnet.
Tailscale app on your phone → use your home model from anywhere in the world.
It's the use case that unlocks real mobility.
Mobile app, remote access, mobility.
Principle of least privilege, MagicDNS, and basic ACLs; never expose it to the public internet.
Keeps the private network truly private.
ACLs, MagicDNS, least privilege.