E1 · Capture
5 minRun the "capture the app" block.
Done when: captura: 0, the five photos from 00-home.png to 04-resultado.png, and passos no steps.json: 5.
Lesson 3 · 20 min
App demo, click by click
To show an app, the real screen is more convincing than any animation. video-demonstrativo drives a browser through your app, takes a photo at each step with the position of the clicked element, narrates with a local voice, and assembles the video with cursor and zoom. In this lesson you do all of it, with a sample app, at no credit cost.
Start the lesson
In this lesson you will
- 1The actions.json: URL, screen size, and one step per click
- 2Capture with agent-browser: a screenshot and the target position for each step
- 3Local narration per step, with the Kokoro voice, no external service
- 4The generated composition and the render with the HyperFrames version pinned
Goal: capture an app step by step with the agent and render the narrated demo that passes the checker.
01From the live app to the MP4
Click each step.
Click each block in the diagram to see what it does.
Steps of the video-demonstrativo skill at commit a87b382 (Oct 5, 2026). Capture before animating: the video shows the real screen.
02The app and the click script
A one-page app in the kit (how many words fit in a narration) and the actions.json with five steps.
File · kit/exemplos/a3-demo/actions.json
{
"url": "http://127.0.0.1:18791/",
"viewport": [1280, 800],
"window": { "top": 96, "titleH": 52, "urlLabel": "127.0.0.1:18791" },
"eyebrow": "DEMONSTRAÇÃO · MINUTOS DE VÍDEO",
"ctaNarration": "Pronto: agora você sabe quantas palavras cabem no seu vídeo.",
"ctaCaption": "Roteiro do tamanho certo",
"steps": [
{ "id": "home", "intro": true, "caption": "Minutos de vídeo · do zero", "narration": "Este app diz quantas palavras cabem numa narração. Vamos usar em três cliques." },
{ "id": "minutos", "do": { "type": "setValue", "selector": "#min", "value": "3" }, "target": "#min", "caption": "1 · Duração: 3 minutos", "narration": "Primeiro, a duração do vídeo: três minutos." },
{ "id": "ritmo", "do": { "type": "setValue", "selector": "#ppm", "value": "140" }, "target": "#ppm", "caption": "2 · Ritmo: 140 palavras por minuto", "narration": "Depois, o ritmo da fala. Uma narração calma fica perto de cento e quarenta palavras por minuto." },
{ "id": "calcular", "do": { "type": "clickText", "tag": "button", "text": "Calcular" }, "target": { "tag": "button", "text": "Calcular" }, "click": true, "caption": "3 · Calcular", "narration": "Agora é só clicar em calcular." },
{ "id": "resultado", "target": "#res", "zoom": true, "caption": "Cabem 420 palavras", "narration": "Cabem quatrocentas e vinte palavras. Esse é o tamanho do seu roteiro." }
]
}03Capture the app
The block serves the app on a local port, runs the capture in its own agent-browser session and shuts everything down at the end.
Terminal · capture the app
K=~/projetos/formacao-cinema-ia-fabrica-v72/kit
command -v agent-browser > /dev/null || { echo "FALTA agent-browser: npm i -g agent-browser@0.6.0 && agent-browser install"; exit 1; }
E=${TMPDIR:-/tmp}/ensaio-fabrica/a3; rm -rf "$E"; mkdir -p "$E"; cd "$E"
git clone -q https://github.com/inematds/video-demonstrativo.git && git -C video-demonstrativo checkout -q a87b382
cp video-demonstrativo/skills/video-demonstrativo/scripts/capture.mjs "$K/exemplos/a3-demo/actions.json" .
python3 -m http.server 18791 --bind 127.0.0.1 --directory "$K/exemplos/a3-demo" > servidor.log 2>&1 & S=$!
until curl -s -o /dev/null http://127.0.0.1:18791/; do sleep 0.3; done
export AGENT_BROWSER_SESSION=ensaio-fabrica
node capture.mjs actions.json > captura.log 2>&1; echo "captura: $?"
agent-browser close > /dev/null 2>&1; kill $S
ls assets/shots
python3 -c "import json; s=json.load(open('steps.json')); print('passos no steps.json:', len(s['steps']))"Example from the repository
captura: 0 00-home.png 01-minutos.png 02-ritmo.png 03-calcular.png 04-resultado.png passos no steps.json: 5
One screenshot per step and the steps.json with the measured boxes. The until curl waits for the port to respond before opening the browser; a fixed sleep fails on a loaded machine.
The block stops at the first line with FALTA agent-browser: npm i -g agent-browser@0.6.0 && agent-browser install. The package is Apache-2.0 (npm, Oct 19, 2026); the install downloads the browser. The session AGENT_BROWSER_SESSION=ensaio-fabrica avoids touching another session you may have open.
04Narrate and assemble
Creates the HyperFrames project, copies screenshots, fonts and steps.json, generates one narration line per step and writes the composition.
Terminal · narrate and assemble
export HYPERFRAMES_NO_TELEMETRY=1 NO_COLOR=1
E=${TMPDIR:-/tmp}/ensaio-fabrica/a3; S=$E/video-demonstrativo/skills/video-demonstrativo
cd "$E" && rm -rf demo
npx -y hyperframes@0.8.176 init demo --non-interactive > init.log 2>&1; echo "init: $?"
cd demo && mkdir -p assets/audio assets/txt
cp -r ../assets/shots assets/ && cp -r "$S/assets/fonts" assets/ && cp ../steps.json .
cp "$S/scripts/composition-template.mjs" build-demo.mjs
node -e 'const d=require("./steps.json"); const l=d.steps.map(s=>s.narration); l.push(d.ctaNarration);
l.forEach((t,i)=>require("fs").writeFileSync(`assets/txt/s${i+1}.txt`, t.trim()+"\n")); console.log("falas:", l.length)'
for f in assets/txt/s*.txt; do
n=$(basename "$f" .txt)
npx -y hyperframes@0.8.176 tts "$f" --voice pf_dora --lang pt-br --speed 0.98 --output "assets/audio/$n.wav" > /dev/null 2>&1 || echo "falhou $n"
done
ls assets/audio | wc -l
node build-demo.mjs | head -1Example from the repository
init: 0 falas: 6 6 index.html gerado · 1920x1080 · TOTAL=30.389s · 6 cenas (5 passos + CTA)
Six narration lines (five steps and the closing), six WAVs. The TOTAL comes from the sum of the lines plus a gap before and after each one: change the line, and the duration changes.
The skill at commit a87b382 calls npx hyperframes without a version, and the old course pinned 0.8.66. The blocks in this course use hyperframes@0.8.176 (npm, Oct 19, 2026) and HYPERFRAMES_NO_TELEMETRY=1. The why, and the rest of HyperFrames, is in course 3.
05Render and check
Check, draft render and the checker, using the duration the composition itself declares.
Terminal · render and check
export HYPERFRAMES_NO_TELEMETRY=1 NO_COLOR=1
K=~/projetos/formacao-cinema-ia-fabrica-v72/kit
cd ${TMPDIR:-/tmp}/ensaio-fabrica/a3/demo
npx -y hyperframes@0.8.176 check 2>&1 | grep -E "◇|✗" | head -3
npx -y hyperframes@0.8.176 render --quality draft --workers 2 --quiet --output renders/demo.mp4 > render.log 2>&1
echo "render: $?"
D=$(grep -o 'id="root"[^>]*data-duration="[0-9.]*"' index.html | grep -o "[0-9.]*\"$" | tr -d \")
echo "duração pedida: $D s"
python3 "$K/conferir.py" renders/demo.mp4 "$D" 1920x1080 --audio; echo "saída: $?"Example from the repository
◇ 0 errors, 0 warnings ◇ 0 issues across 9 sample(s) ◇ 0 errors, 0 warnings render: 0 duração pedida: 30.389 s OK: 30,4 s · 1920×1080 · 30 fps · com áudio saída: 0
On this machine the 30 s render took less than a minute (measured outside the capture; software render, --workers 2). The requested duration comes from the root's data-duration, so the checker works for any script of yours.
06Practice
From the sample app to your app.
E2 · Rendered demo
10 minRun "narrate and assemble" and then "render and check".
Done when: index.html gerado · 1920x1080, render: 0 and OK: … · 1920×1080 · 30 fps · com áudio with saída: 0.
E3 · Your app
30 minAsk your agent for an actions.json with 4 to 6 steps for an app of yours that runs on localhost. Swap the URL, run the capture with your file, and then the next two blocks.
Done when: In the dry-run folder, python3 -c "import json; print(len(json.load(open('steps.json'))['steps']))" → the number of steps in your actions.json, and the render ends with OK: … · com áudio and saída: 0.
07Lesson quiz
The explanation appears after you answer.
Question 1 of 3
Why is the viewport in actions.json fixed?
Correct
The cursor aims at the box measured with getBoundingClientRect. If the capture screen and the photo have different sizes, the click shows up outside the button.Not that one
The cursor aims at the box measured with getBoundingClientRect. If the capture screen and the photo have different sizes, the click shows up outside the button.Review: From the live app to the MP4Question 2 of 3
What sets the final length of the demo?
Correct
build-demo measures each WAV and builds the scene around the speech. Longer speech, longer scene.Not that one
build-demo measures each WAV and builds the scene around the speech. Longer speech, longer scene.Review: The app and the click scriptQuestion 3 of 3
The block waits for the app's port with an until curl loop. Why not sleep 1?
Correct
Waiting for the state (the port responds) and not for the clock is what makes the block repeatable on any machine.Not that one
Waiting for the state (the port responds) and not for the clock is what makes the block repeatable on any machine.Review: The app and the click scriptQuiz complete
You got {a} of {n} right. Get them all right and the lesson is marked as done.Pick an option. The explanation appears after you answer.
08In three sentences
Capture before animatingagent-browser photographs the real app and measures where to click.
Speech sets the timingeach step lasts as long as its narration, made with a local voice.
Pinned versionHyperFrames 0.8.176 in the blocks, checker on the MP4.