PTENES
Skip to content
MODULE 4.5

🗣️ Step-by-step #3: your local buddy with a name, face, and voice

Now we bring everything in the course together in one place: a local personal assistant with its own personality (system prompt), a face (interface), a voice (TTS), and even ears (STT). It runs on your machine, remembers you (RAG), and stays with you on your phone (Tailscale). It’s the video “give your model a face, a name, and a voice” turned into a step-by-step project.

6
Topics
~40
Minutes
Advanced
Level
Practical
Type

Detailed content

1

🎯 The goal: a real buddy

You already have the pieces scattered throughout the course: a running model (3.1), a local API (3.5), memory with RAG (3.6), and a private network with Tailscale (2.4). This step-by-step guide brings it all together into a personal companion — with a name, a way of speaking, a screen to chat on, and a voice to answer you out loud. Nothing leaves your machine.

🆕 New here?

"System prompt" is a fixed instruction that the model reads before for any conversation: it defines who it is, its tone, and its rules. It’s the personality of the buddy. "TTS" = text-to-speech (text → speech) and "STT" = speech-to-text (speech → text): both also run locally, without sending your audio to anyone.

🔑 The four pieces of the buddy

A buddy = four things you’ve already seen, put together:

  • •Personality: a system prompt in the Modelfile with a name, tone, and rules.
  • •Face: an interface — Open WebUI (3.2) or a minimal HTML page.
  • •Voice and ears: Piper (TTS) to speak and whisper.cpp (STT) to listen.
  • •Memory + reach: RAG (3.6) to remember you and Tailscale (2.4) to use it from your phone.

✓ Local Buddy

  • ✓Personality and rules 100% under your control
  • ✓Chat, voice, and memory without leaving your machine
  • ✓Unlimited use — talk all day, only pay for electricity
  • ✓Accessible from your phone through your private network

✗ Cloud assistant

  • ✗Personality limited to what the platform allows
  • ✗Your audio and conversations pass through third parties
  • ✗Voice and heavy use are billed per minute/token
  • ✗Disappears if access, pricing, or regional availability is cut off
Name
system prompt
Face
interface
Voice
Piper TTS
Memory
RAG + network
2

🪪 Personality via system prompt

Personality doesn't come from "training" anything — it comes from a fixed instruction. In Ollama, you save that instruction in a Modelfile (the recipe you saw in module 3.1) and creates a named model from it. Change the text in SYSTEM and you change who the buddy is.

⌨️ Copy-run: give "Hermes" a name and tone

Objective: create a local model called hermes with a defined personality. Save as Modelfile and run the commands.

# Modelfile — defines "Hermes," your local buddy
FROM qwen2.5:7b-instruct
# the system prompt = the buddy's fixed personality
SYSTEM """
You are Hermes, Nei’s local personal assistant.
Tone: direct, warm, and lightly humorous. Address the reader as “you.”
Rules: short answers by default; ask before making assumptions;
never make up data—if you don't know, say so. Respond in Brazilian Portuguese.
"""
# fine-tune the behavior
PARAMETER temperature 0.6
PARAMETER num_ctx 8192
# create the model from the recipe and chat
ollama create hermes -f Modelfile
ollama run hermes

How to check: ask "who are you?". It should introduce itself as Hermes, in the set tone—and follow the rules (short, no making things up). Don't like it? Edit the SYSTEM and run ollama create again.

💡 Practical tip

A good system prompt has four parts: who it is (name/role), how it speaks (tone), what it can/can’t do (rules), and the language. Avoid huge paragraphs—the model follows short, specific instructions better than a manual. Start simple and adjust as you see how it responds.

Role
who it is
Tone
how it speaks
Rules
what matters
Language
PT-BR
3

🖥️ One interface: a dedicated interface

"Giving it a face" just means pointing an interface at your model. Two paths: the ready — reuse Open WebUI (3.2), which already has chat with history — or lean: a 20-line HTML page that talks directly to the local API (3.5). Start with the simplest one to understand the flow; then move up to Open WebUI when you want history and attachments.

⌨️ Copy-run: a minimal "face" in HTML

Objective: a page that sends your question to the local endpoint and displays the answer. Save it as buddy.html.

<!-- buddy.html — talks to the local model -->
<!doctype html><meta charset="utf-8">
<h2>🤝 Hermes</h2>
<input id="q" style="width:70%" placeholder="ask...">
<button onclick="send()">Send</button>
<pre id="out"></pre>
<script>
async function send(){
  const q = document.getElementById('q').value;
  const r = await fetch(
    'http://localhost:11434/v1/chat/completions',{
    method:'POST', headers:{'Content-Type':'application/json'},
    body: JSON.stringify({ model:'hermes',
      messages:[{role:'user', content:q}] }) });
  const j = await r.json();
  document.getElementById('out').textContent =
    j.choices[0].message.content;
}
</script>

How to check: open the file in your browser, type a question, and click Send — your local model’s response appears on screen. It’s your buddy with a personality of its own.

⚠️ Attention: CORS

If the browser blocks the call (CORS error when opening via file://), allow the origin in Ollama: run it with OLLAMA_ORIGINS=* (e.g., OLLAMA_ORIGINS=* ollama serve). For a real buddy, prefer the Open WebUI in http://localhost:3000 — it already takes care of that and gives you history, model switching, and document RAG.

✓ Open WebUI (ready to use)

  • ✓Conversation history and multiple chats
  • ✓Model switching and embedded-document RAG
  • ✓Ready to use from your phone over the tailnet

📄 Minimal HTML (for teaching)

  • •Shows the whole workflow in 20 lines
  • •Easy to customize its "look" however you want
  • •Without history or attachments — it’s just the skeleton
4

🔊 Giving it a voice: local TTS (Piper)

Voice is what turns a chat into a buddy. Instead of just reading on the screen, it speaks. For this, we use a TTS engine that runs on your machine—the Piper — and streams the model’s response directly to the speaker.

🆕 New here?

"TTS" = text-to-speech: converts text to speech. O Piper is a lightweight, fast local TTS (it even runs on CPU), with several voices — including in Brazilian Portuguese. Each voice is a .onnx; replace the file and the buddy changes its voice.

The voice pipeline is short — text goes in one end, sound comes out the other:

1

The response becomes text

The local model responds via API (3.5). You take only the response text.

2

Piper synthesizes speech

The text goes into Piper with a voice .onnx; it generates an audio file .wav.

3

The system plays the audio

A simple player (aplay on Linux, afplay on macOS) reproduces the .wav. The buddy “spoke.”

⌨️ Copy-run: the buddy says a sentence

Objective: install Piper and turn text into sound through the speaker. Paste it into the terminal.

# install Piper (local TTS)
pip install piper-tts # ou baixe o binário do repo
# download a PT-BR voice (.onnx) from the Piper repository
# text -> Piper -> .wav file -> speaker
echo "Olá, eu sou o Hermes." \
  | piper --model pt_BR-faber-medium.onnx \
      --output_file fala.wav
aplay fala.wav # macOS: afplay fala.wav

How to check: you hears the sentence. It changed the .onnx? The voice changes. That same piper ... | aplay is the voice component of the loop in topic 6.

5

🎤 Listen: local STT (optional)

Giving it a voice is already halfway there. The other half is giving it ears: being able to talk to the buddy instead of typing. It’s optional—but with local STT, the loop is complete (you speak → it understands → responds → speaks back).

🆕 New here?

"STT" = speech-to-text: converts speaks in text (the opposite of TTS). The whisper.cpp is a lightweight local implementation of the Whisper model that runs on the CPU—it records from the microphone, transcribes, and returns text without sending your audio to the cloud.

⌨️ Copy-run: transcribe your voice locally

Objective: record 5 seconds from the microphone and see the text of what you said. All offline.

# whisper.cpp = local STT (speech -> text), runs on CPU
git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp && make
./models/download-ggml-model.sh small # baixa o modelo
# record 5s from the microphone and transcribe in PT-BR
arecord -d 5 -f S16_LE -r 16000 fala.wav
./main -m models/ggml-small.bin -l pt -f fala.wav

How to check: the text of what you said appears in the terminal. This text is exactly what you would pass as content in the API call — closing the voice loop.

✓ Worth listening when

  • ✓You want to use it hands-free (cooking, driving)
  • ✓Speaking is faster than typing for your use case
  • ✓You want a 100% private voice assistant

✗ Stick to text if

  • ✗Hardware is underpowered — STT + LLM + TTS together are demanding
  • ✗A noisy environment interferes with transcription
  • ✗You just want the text buddy for now (start simple)
6

🧠 Memory, voice, and phone: all together

We still need to fit in the last two pieces and start the engine. Memory: with RAG (3.6), the buddy remembers you—preferences, projects, past conversations—by retrieving those passages and injecting them into the context before responding. Reach: running behind Tailscale (2.4), you can talk to it from your phone, anywhere, as if you were at home. See how it all fits together:

🖥️ Interface interface + 🧠 Brain LLM + system + 🔊 Voice TTS + STT + 🧠 Memory RAG + network = 🗣️ Your buddy name · face · voice · memory

Read it as a equation: face + brain + voice + memory (the cyan pieces you assembled in the course) add up to a full local buddy (in amber). Remove any part and it still works — it just feels less buddy-like.

💡 Memory + mobile, in practice

Memory (RAG, 3.6): store your notes (preferences, facts, history) in a vector store and, with each question, retrieve the relevant passages and add them to the prompt—this way Hermes “remembers” without retraining anything. Cell phone (Tailscale, 2.4): with OLLAMA_HOST=0.0.0.0 and the Tailscale app on your phone, switch localhost by IP 100.x from your machine and talk to the buddy from anywhere—without exposing anything to the public internet.

⌨️ Copy-run main: the talking buddy loop

Objective: the complete circuit in one file — question → local API → response → Piper speaks. Save it as buddy-voz.sh.

#!/usr/bin/env bash
# buddy-voz.sh — question -> local API -> response -> Piper speaks
while true; do
  read -rp "you: " question
  [ "$pergunta" = "sair" ] && break
  # 1) send to the local API (Ollama, OpenAI-compatible)
  resposta=$(curl -s http://localhost:11434/v1/chat/completions \
    -d "{\"model\":\"hermes\",\"messages\":[{\"role\":\"user\",
    \"content\":\"$pergunta\"}]}" \
    | jq -r '.choices[0].message.content')
  echo "hermes: $resposta"
  # 2) Piper speaks the response
  echo "$resposta" \
    | piper --model pt_BR-faber-medium.onnx --output_file r.wav
  aplay r.wav
done

How to check: run bash buddy-voz.sh, ask something and listen your buddy's response. Type exit to shut down. Requires curl, jq, piper e aplay. Want audio too? Switch the read using the whisper.cpp transcription (topic 5).

✓ A well-built Buddy

  • ✓Short, clear, tested system prompt
  • ✓Everything local: voice, memory, and data on your machine
  • ✓Access only through the tailnet, never the open internet

✗ Common errors

  • ✗Expose port 11434 directly to the internet
  • ✗Huge system prompt that the model ignores
  • ✗Stacking STT+LLM+TTS on hardware that can’t handle it

Self-check (optional): what gives your local buddy the personality — name, tone, and rules?

Name
SYSTEM
Face
UI / HTML
Voice
Piper / Whisper
Always
RAG + tailnet

📌 Module summary

✓
Personality = system prompt — name, tone, and rules in the Modelfile’s SYSTEM, created with ollama create.
✓
Face = interface — Open WebUI ready to use or a minimal HTML page that talks to the local API.
✓
Voice and ears = local TTS + STT — Piper speaks, whisper.cpp listens, all offline.
✓
Memory + reach — RAG (3.6) to remember you and Tailscale (2.4) to use it from your phone.

Next module:

4.6 — Performance, scale & troubleshooting

← Previous module Back to the track Next module →