PTENES
MODULE 2.4

🖥️ Computer Actions

"Browser, bash, real cursor." Because it lives on your PC, Hermes doesn’t just answer—it acts: creates files, navigates with screen vision, runs commands, and moves the cursor for real. Real power that calls for responsibility.

6
Topics
22
Minutes
Interm.
Level
Practice
Type
1

📂 Create and Organize Files on Your Desktop

Like Hermes runs on your machine (Track 1), it has access to the file system: it creates, edits, moves, and organizes files on the desktop. It goes far beyond "answering a question" — it produces real artifacts that are saved on your computer.

🖥️ Hermes on your PC 📂 Create a File 🌐 open browser ⌨️ run bash 🖱️ move cursor task completed

💡 Practical tip

Acting on your PC is useful for tasks you’d do by hand: organizing a downloads folder, generating a file from a conversation, renaming batches. Start with reversible tasks.

2

🌐 Browser vision — Native DevTools

Hermes browses using Native Chrome DevTools e visual screen reading. It “sees” the page as you do — it doesn't rely only on raw HTML; it understands the rendered layout. That's what lets it act on sites that don't have an API.

illustrative recreation — not a real screenshot

› hermes: opening browser (browser vision)…

› reading screen: I identified the “search” field

› typing: “flight to Dubai → Toronto”

› clicking the “filter by price” button 🖱️

✓ 3 options found, opening the cheapest

📊 Why “vision” and not just HTML

  • Works even on JavaScript-heavy sites
  • Understands buttons and menus that raw HTML doesn't reveal
  • Brings the agent closer to human browsing behavior
3

⌨️ Bash — real terminal

Hermes runs commands in the terminal (bash)That gives you real automation power: installing things, moving files in batches, running scripts. It’s the same access point you use to store API keys in the env (module 2.3).

Bash usage example (illustrative)

# "organize meus downloads por tipo"
mkdir -p ~/Downloads/{imagens,pdfs,zips}
mv ~/Downloads/*.png ~/Downloads/imagens/
mv ~/Downloads/*.pdf ~/Downloads/pdfs/
mv ~/Downloads/*.zip ~/Downloads/zips/

⚠️ Common mistake

Requesting a destructive action without reviewing it (e.g., rm -rf). Bash has no "undo." Prefer having the agent show you the command before running it, especially when it deletes or overwrites something.

4

🖱️ Real cursor — not just a screenshot

The difference many tools lack: Hermes moves the cursor for real and clicks — real agency, not just taking a screenshot. Combined with browser vision and bash (“fused”), it operates any app, even without an API.

✗ Screenshot only

  • ✗Sees the screen, but can’t click
  • ✗Depends on you taking the action
  • ✗"Tells you how to do it," doesn’t do it

✓ Real cursor (fused)

  • ✓Sees AND clicks AND types
  • ✓Completes the task on its own
  • ✓It's an agent, not an advisor

💡 Connects to Trail 1

This is exactly the concept of "agent, not chatbot": the real cursor is physical proof that Hermes does, not just talks.

5

🏃 Example: on your way to the gym

Real-world scenario: you left home, you're on your way to the gym, and you remember you forgot to do something on the computer. You send a message from your phone — and Hermes runs on your PC, even when you're far away.

1

You (on your phone)

"I forgot to save that report in the client’s folder. Can you do that for me?"

2

Hermes (on your PC)

Finds the file, creates/opens the right folder, moves it, and confirms—all using bash + vision.

3

Result

"Done. Moved to Clientes/Acme/Relatórios." You didn’t even stop walking.

📊 Why this is powerful

Combines "one brain, 22 mouths" (you talk through your phone) with "actions on the computer" (it runs tasks on the PC). The input channel and the place where actions happen are decoupled.

6

⚠️ Enable responsibly

Computer actions are enabled in the tools of Hermes. It’s a powerful feature — and power calls for care. The principle of less access (Track 3, module 3.1) applies here: share only what's necessary.

✓ Responsible use

  • ✓Start with reversible actions
  • ✓Review destructive commands beforehand
  • ✓Enable only the tools you’ll use

✗ Risk indicators

  • ✗Allow everything "to make things easier"
  • ✗Let it act without any review
  • ✗Mixing with automatic email sending

💡 Rule of thumb

"With great power comes great responsibility." The more your agent can do on your PC, the more important it is to grant the least access. Trail 3 goes deeper into security.

📌 Module Summary

✓
Files — creates and organizes on the desktop, with system access.
✓
Browser vision — Native DevTools + visual reading of the screen.
✓
Bash — real automation; be careful with destructive commands.
✓
Real cursor — moves and clicks for real (fused), not just screenshots.
✓
Responsibility — enable it in tools with less access (see 3.1).

Next Module:

2.5 — MCPs: the instruction manual