Detailed content
🔌 Why a local API
A chat interface is great for talking with you. But a program doesn't communicate through a screen—it needs an address to send text and receive a response. That address is the API. Ollama’s trick is to speak exactly the same “format” OpenAI uses: that way, any app, script, or agent that already knows how to talk to ChatGPT works with your local model by changing just one line.
🆕 New here?
"API" (Application Programming Interface) is an entry point through which one program asks another for something. "Endpoint" is the exact address of that port — a URL like http://localhost:11434/v1/chat/completions. "OpenAI-compatible" means that the request and response format is identical to OpenAI’s API — same keys model, messages, choices — just served by your machine.
🔑 The central idea
Compatibility is the lever. With it, you get three things at once:
- •Reuse: libraries, SDKs, and tools built for OpenAI work without rewriting anything.
- •Portability: switch between cloud and local by changing only the base_url — the code stays the same.
- •Automation: scripts and agents (Track 4) start using the model without you opening any screens.
✗ No API (chat on screen only)
- ✗Every use requires you to type by hand
- ✗No automation or scheduling
- ✗Impossible to plug into other programs
- ✗No 24/7 agents
✓ With a local API
- ✓Any script can call the model over HTTP
- ✓Same OpenAI SDK, model on your machine
- ✓Foundation for automations, RAG, and agents
- ✓Everything offline, with no per-token fees
🌐 The Ollama endpoint
When you installed Ollama (Module 3.1), it already started an HTTP server listening on http://localhost:11434. localhost = the machine itself; 11434 = the port. This server offers two sets of routes: the native from Ollama and the OpenAI-compatible under /v1. To reuse existing apps and SDKs, always use the ones for /v1.
The path of a request, from your program to the model and back:
Your program builds the request
A JSON with the model and the list of messages — exactly the OpenAI format.
The Ollama server receives requests at :11434
It loads the model into memory (if it isn’t already there) and runs inference locally.
The response comes back as JSON
In the same format as OpenAI: the text comes in choices[0].message.content.
🗺️ The routes that matter
Objective: learn the addresses. The ones for /v1 are the ones you use with ready-made SDKs.
How to check: run curl http://localhost:11434/v1/models — should list the models you’ve already downloaded in JSON.
💡 Practical tip
If the port 11434 doesn't respond, the server isn't running. On Linux/macOS, the Ollama app keeps it running; if you need to start it manually, use ollama serve. "Connection refused" is almost always this: the service isn't running or is on another machine.
⌨️ First call with curl
The fastest way to prove the API works is the curl — a terminal tool that makes HTTP requests. No library installation, no code: just the terminal and a JSON object. If this responds, your local API is up.
⌨️ Practical example: chat via curl
Objective: send a message to the model and receive the response in JSON. Paste it into the terminal (with Ollama running and the model qwen2.5:7b already downloaded).
How to check: the output is a JSON object with a field choices. The response text is in choices[0].message.content. If it appears, you’ve just served your model through an API.
📊 Anatomy of the response JSON
- model — which model responded.
- choices[0].message.content — the generated text (what you want).
- choices[0].finish_reason — why it stopped (e.g., stop).
- usage — input and output token counts.
🐍 From code: OpenAI SDK pointed to the local model
This is where compatibility shines. The Official OpenAI SDK (the library openai in Python) works with Ollama—just change the base_url for your local endpoint and pass a api_key any (Ollama doesn't validate the key; it's required only because the SDK asks for one).
🐍 Practical example: chat in Python
Objective: call the local model through the OpenAI SDK. Install with pip install openai and run.
How to check: the script prints the sentence generated by the model. To switch back to the cloud, change base_url and the key—the rest of the code stays identical. That's the whole point.
💡 Practical tip
In JavaScript/TypeScript, it’s the same story: the package openai accepts baseURL: "http://localhost:11434/v1" e apiKey: "ollama". Store the base_url in an environment variable (OPENAI_BASE_URL) to switch between local and cloud without touching the code.
✓ Do this
- ✓Use the model name shown in ollama list
- ✓Centralize base_url an env var
- ✓Handle timeouts: the first request loads the model (slower)
✗ Avoid
- ✗Forget the /v1 at the end of base_url
- ✗Using a model name that hasn’t been downloaded
- ✗Put a real OpenAI key in the local code
🌊 Streaming, embeddings, and multiple models
The same API does three more things you'll use a lot: streaming (the response arrives word by word, like in ChatGPT), embeddings (turn text into vectors—the foundation of RAG, in the next module) and route by model (just change the field model).
🌊 Streaming: token-by-token responses
Objective: display the text as it’s generated, without waiting for it to finish. Just stream=True.
How to check: the text appears gradually, not all at once. In curl, the equivalent is to add "stream": true in the JSON body.
🧮 Embeddings: text becomes a vector
Objective: generate a text’s meaning vector—the building block of RAG (Module 3.6). Use an embedding model such as nomic-embed-text (download with ollama pull nomic-embed-text).
How to check: the response contains data[0].embedding — a long list of numbers. That’s what a vector database stores to search by similarity.
🔀 Multiple models, one endpoint
You don’t launch a server for each model. The same endpoint serves all the models you’ve downloaded—just change the field model: a small, fast model for classification, a larger one for reasoning, and an embedding model for RAG. Ollama loads and unloads them as needed.
🔒 Safely exposing it on the tailnet
By default, the API only responds on localhost — no one outside can reach it. But in Module 2.4, you set up a tailnet (private network via Tailscale). Put the two pieces together, and you can use your home model from your work laptop or phone—all within your private network, without exposing anything to the internet.
Read from left to right: the app/script (purple) calls Local API on port 11434, which runs the model. The branch at cyan shows another device reaching the same API over the tailnet — without exposing anything to the public internet.
⌨️ Practical example: open it on the tailnet
Objective: make Ollama listen on every interface (not just localhost) so other devices on your tailnet can reach it.
How to check: from the second device, the curl with the IP 100.x should respond in JSON. If it doesn't, check whether both are on the tailnet (tailscale status).
⚠️ Alert: never expose it to the public internet
OLLAMA_HOST=0.0.0.0 opens the port to any network the machine is connected to. Within the tailnet this is safe: only your authenticated devices can see the 100.x. On the open internet, no. The Ollama API has no login or password—anyone who can reach the port can use (and abuse) your model.
- ✗NEVER port-forward 11434 on your router.
- ✗NEVER point the port to a public IP.
- ✓Access remotely ONLY through the tailnet (IP 100.x) — that’s the secure way.
✓ Tailnet (secure)
- ✓Your own authenticated devices only
- ✓End-to-end encrypted traffic
- ✓No ports open on the router
✗ Public internet (dangerous)
- ✗Anyone on the web can reach the port
- ✗No login: completely open API
- ✗Hijacked model, your electricity bill
Self-check (optional): to use the OpenAI SDK with your local model, what do you adjust?
📌 Module summary
Next module:
3.6 — Embeddings, memory & RAG