Foundations: how agents use a website today|INEMA.CLUB-PRO

In this lesson you will

  1. 1Role and name: the two things the agent looks for
  2. 2The page tree printed by the kit
  3. 3The gaps: a control with no name, a div pretending to be a button
  4. 4Tree as text × screen as pixels

Goal: read the accessibility tree the agent receives and compare it with the screenshot of the same page.

01From page to click

Click each block.

Click each block in the diagram to see what it does.

Anything that isn't in the tree with a role and a name, the agent can only find by guessing from the screen.

02How to read a tree

Three questions for each line.

1. What's the role?

textbox is a text field, combobox is a list of options, button is a button, link is a link, heading is a title, img is an image. The role comes from the HTML element: input, select, button, a, h1, img.

A div with onclick doesn't become a button: it shows up as loose text.

03The tree for antes.html, next to the screen

arvore.mjs opens the page in a headless Chromium, prints the tree with Playwright 1.63.0's locator.ariaSnapshot() (the same snapshot format Playwright MCP uses), saves the screenshot, and checks what's missing.

Terminal · tree and screen
cd ~/projetos/formacao-webmcp-base-v72/kit
node arvore.mjs antes --tela ${TMPDIR:-/tmp}/tela-antes.png; echo "saída: $?"
Example from the repository
## árvore de acessibilidade · 9 linhas · 182 bytes
- text: Agendar revisão
- img
- textbox "Seu nome"
- combobox:
  - option "Escolha" [selected]
  - option "Revisão básica"
  - option "Revisão completa"
- textbox
- text: Agendar
## tela · /tmp/tela-antes.png · 14536 bytes (o modelo teria de ler pixels)
## conferência · http://127.0.0.1:PORTA/antes.html
  - img sem nome acessível
  - combobox sem nome acessível
  - textbox sem nome acessível
  - <html> sem lang
  - sem <main>
  - sem <h1>
  - <div> clicável que não é botão ("Agendar")
  - campo #nome sem <label> (só placeholder)
  - campo #servico sem <label>
  - campo #data sem <label>
problemas: 10
saída: 1

Nine lines of tree versus 14,536 bytes of PNG. In the tree, the service list (combobox), the date field, and the image have no name, and the "Agendar" button became text. The check lists 10 problems and the script exits with 1.

04Practice

The tree of the example page and the tree of your site.

E1 · Tree × screen

10 min

Run the "tree and screen" block. Open /tmp/tela-antes.png and, for each line of the tree, find the control in the image. Mark the ones that have no name.

Done when: node arvore.mjs antes --tela /tmp/tela-antes.png → problemas: 10 and saída: 1, and the file /tmp/tela-antes.png exists.

E2 · The tree of your site

10 min

Run arvore.mjs on the same page of your site that you used in lesson 1. Write down the number of problems: it comes back in lesson 3 and in the plan.

Done when: node arvore.mjs https://[your-site]/[page] ends with problemas: N and saída: 0 (N = 0) or saída: 1, with N noted down.

05Lesson quiz

Question 1 of 4

What does Playwright MCP hand to the model so it can decide where to click?skill

06In three sentences

Tree, not pixelsThe browser agent reads roles and names, a short list of text.
No quotes, no nameA control with no name is a control the agent has to guess.
Placeholders misleadIt gives a name in the tree, but it is not a label; use label.

Going further

Previous lessonNext lesson