Detailed content
🚀 Why start with Ollama
Running an LLM “by hand” involves downloading weights, compiling libraries, and adjusting a thousand flags. The Ollama solve this: it’s a runtime that downloads, manages, and runs local models with a single command. You type ollama run and, in seconds, you’re chatting. It’s the least friction possible between you and your first result.
🆕 New here?
"Runtime" is the program that loads the model into memory and generates text (called “inference”). Without it, the weights are just a file sitting on disk.
"Ollama" is one of these runtimes — designed to be easy to use. Under the hood, it uses llama.cpp (the engine that runs the model on CPU and GPU), but hides all the complexity behind simple commands on macOS, Linux, and Windows.
Read from left to right: you install once, then just download (pull) e run as many models as you want. The blue band highlights the midpoint: is all local.
✓ With Ollama
- ✓One command downloads and runs the model
- ✓Automatically uses the GPU when available
- ✓Same workflow on Mac, Linux, and Windows
- ✓Comes with a local API ready to use
✗ Compiling by hand
- ✗Download the weights and convert the formats yourself
- ✗Compile libraries and get the drivers right
- ✗Dozens of flags before the first “hello”
- ✗Great for learning — terrible for getting started
📥 Install with 1 command
Installation is the shortest part of this module. In macOS and Linux is a single command pasted into the terminal; in Windows is a clickable installer. Here are the three steps:
Run the installer
On Mac/Linux, paste the command below. On Windows, download the app from ollama.com/download and click Install.
Confirm that it was added
Run ollama --version. If a version number appears, you’re all set.
The service is already running
Ollama starts a background service that listens on localhost:11434. It’s the local API we’ll use later (module 3.5).
⌨️ Copy and run: install Ollama
Objective: to leave the command ollama available on your machine.
How to check: the script finishes without errors and, at the end, suggests running a model. If you have an NVIDIA GPU, it detects CUDA automatically. Move to the next block to confirm the version.
⌨️ Copy and run: confirm the installation
Objective: prove that Ollama is installed and responding.
How to check: a line appeared with ollama version is ...? Then the engine is running. If you get "command not found," close and reopen the terminal (or, on Windows, restart the session).
💡 Practical tip
That | sh at the end of the command means “run the script that was just downloaded.” It’s Ollama’s official pattern, but it’s always good practice to open the URL in your browser and take a look at what a script does before piping it to the shell.
💬 Your first chat
Time for the “wow” moment. A single command downloads a model (if it isn't already on disk) and opens a chat prompt right in the terminal. Start with the llama3.2 — it’s lightweight, downloads quickly, and runs well on most machines.
⌨️ Copy and run: open the chat
Objective: download a model and chat with it now.
How to check: the first time, it shows a download bar; afterward, a prompt appears >>>. Type, for example, "explain what local AI is in 2 sentences" and read the answer coming from your own machine. To exit, type /bye.
📊 What’s happening
- Without internet after the download: downloaded once, it runs offline forever.
- The first response may take a little while — is the model loading into memory. The next ones come faster.
- Nothing is sent outside: your prompt and the response stay only on your computer.
Optimal balance. Start here if you're unsure.
More "intelligent," good at coding and reasoning. Requires more RAM/VRAM.
Tiny and fast. Saves the day on modest machines.
💡 Practical tip
Frozen or too slow? Exit with /bye and try a smaller model (the gemma2:2b (almost always runs). Model size vs. memory is covered in topic 5—and there you'll learn how to choose without guessing.
🗂️ Managing models
Over time, you’ll collect a few models. Four commands handle almost everything: pull low, list shows what you have, ps shows what’s running now and rm delete to free up disk space.
⌨️ Copy and run: the 4 everyday commands
Objective: download, list, inspect, and remove models.
How to check: run ollama list and see the table with the name, size, and date. After a ollama rm, run list again — the model disappeared and the space came back.
📁 Where the files are stored
Downloaded weights are stored locally, in an Ollama folder:
That's why a model several GB in size only downloads once: next time, Ollama reads it from disk. Keep an eye on storage space — large models add up fast.
📐 The right size for your hardware
That number in the tag— :2b, :7b, :14b — is the model size in billions of parameters. More parameters = more "intelligence," more memory, and more slowness. The beginner's rule of thumb: start small and scale up only when needed.
✓ Start small
- ✓Downloads quickly and runs on the first try
- ✓You feel your machine’s real speed
- ✓Scale up with confidence, knowing the limits
✗ Going straight for the biggest
- ✗Huge download before any results
- ✗May not fit in memory ("out of memory" error)
- ✗Runs sluggishly and makes you think “local is bad”
📊 Quick guide by memory (rule of thumb)
Approximate values, assuming Ollama downloads a "quantized" (compressed) version by default — covered in depth in module 3.3. For the correct memory calculation, see Track 2.
💡 Practical tip
Not sure how much memory you have? Go back briefly to Track 2: on Linux, free -h; with an NVIDIA GPU, nvidia-smi shows the VRAM. This number is exactly the ceiling that determines which tag :Nb fits.
🛠️ Customize with Modelfile
So far, you've used the "out-of-the-box" models. But you can create your own version — with a fixed personality, rules, and tone — without training anything. The secret is the Modelfile: a little text recipe that Ollama turns into a ready-to-use model with a name you choose.
🆕 New here?
"Modelfile" is a text file (similar to a Dockerfile, if you know it) that defines the recipe for your model: which base model to start from (FROM), what fixed instruction to give (SYSTEM) and which parameters to use. Ollama “bakes” this recipe into a new model that you call by name.
⌨️ Copy and run: create “my-assistant”
Objective: create an assistant with a fixed personality from a model you’ve already downloaded.
1) Create a file called Modelfile (without an extension) with this content:
2) In the same folder, turn the recipe into a model:
How to check: run ollama list and see my-assistant in the list. When you chat, it already follows the tone you set — without needing to repeat the system prompt every time. Replace the sections in <...> whatever makes sense for you.
Self-check (optional): which command do you use to download a model without opening the chat?
📌 Module summary
Next module:
3.2 — ChatGPT-style interface: Open WebUI & LM Studio