PTENES
TRACK 2

🎯 Action: Application by Sector

From theory to practice. Supervising autonomous systems, human-robot safety protocols, VR simulators, and industry applications in healthcare, logistics, and agriculture.

TRACK 2 Industry action Supervision Safety + VR Protocols Health Logistics Agriculture Safety Manufacturing Professional Reskilled
8
Modules
48
Topics
~4h
Duration
Intermediate.
Level

Course roadmap

Detailed content

2.1 ~35 min

🧠 Supervising Autonomous Systems: Teaching Judgment

Technical and cognitive skills for supervising autonomous robots with critical judgment, not blind obedience.

What it is:

Skill set for interpreting real-time data from robotic systems: telemetry dashboards, reading failure patterns, and understanding the machine’s operating envelope.

Why learn:

Without technical competence, the supervisor responds to alarms without understanding the root cause. The difference between stopping the line as a precaution and stopping it in a panic is USD 50k per hour in lost production.

Key concepts:

Torque and temperature telemetry, SCADA/HMI dashboards, recognizing normal deviations versus anomalies, operating envelope (maximum load, speed, reach).

What it is:

Mental skills for supervising autonomous systems: calibrating trust in the robot, maintaining situational awareness during monotony, and tolerating ambiguity when the system is in a gray area.

Why learn:

Automation complacency kills. When everything works 99.9% of the time, the human brain switches off during the critical 0.1%. Trust calibration is the antidote.

Key concepts:

Trust calibration (Lee & See 2004), situational awareness (Endsley), tolerance for ambiguity, active vs. passive monotony.

What it is:

A clear decision tree: what the supervisor handles independently, what gets escalated to a maintenance engineer, and what requires an immediate production shutdown.

Why learn:

Escalating too much paralyzes operations. Escalating too little causes accidents. The right protocol balances safety with productivity.

Key concepts:

Alert levels (green/yellow/red), response time for each level, pre-escalation checklist, line-stop authority.

What it is:

A documented phenomenon in which operators overtrust automation, stop actively monitoring it, and lose the ability to intervene when the system fails.

Why learn:

Real cases: Air France 447, Tesla Autopilot, Uber ATG. All involved excessive human trust in autonomous systems. The pattern repeats in factories.

Key concepts:

Ironies of automation (Bainbridge 1983), out-of-the-loop unfamiliarity, skill degradation, alarm fatigue.

What it is:

MERL Research's systematic framework for mapping human competencies in robot collaboration contexts: Operational, Human-Centered, Relational, Cognitive, Systemic.

Why learn:

Provides a standardized language for defining training curricula. Companies that use structured frameworks reduce onboarding time by 40%.

Key concepts:

5 OHRCS domains, gap mapping, integration with ISO 10218, application in corporate certification programs.

What it is:

National Science Foundation mapping of the essential competencies for Human-Robot Collaboration: technical, social, adaptive, and ethical.

Why learn:

The NSF influences university curricula and funding programs. Knowing the mapping helps anticipate what future certifications will require.

Key concepts:

4 NSF pillars (technical, social, adaptive, ethical), alignment with ABET, gap analysis by role (operator, supervisor, engineer).

🔧In practice

  • ✓Read the telemetry (torque, temperature) and learn the operating envelope before taking over the shift
  • ✓Combat complacency: perform active checks even when everything seems normal
  • ✓Follow the escalation tree (green/yellow/red) and know when to stop the line
  • ✓Calibrate your trust in the system — neither blindly nor paranoically
View Full Version
2.2 ~35 min

🛡️ Human-Robot Safety Protocols

From pre-commissioning risk assessment to incident response. ISO standards, adapted LOTO, and a near-miss culture.

What it is:

A systematic process for identifying hazards and assessing risks that MUST be carried out before any robot enters operation. ISO 12100 and ISO 10218 require it.

Why learn:

Without a documented risk assessment, the company operates out of compliance. Beyond the legal risk, it's the basis for defining safety zones, speeds, and contact forces.

Key concepts:

ISO 12100 (general principles), HIRA (Hazard Identification and Risk Assessment), severity × probability matrix, mandatory documentation.

What it is:

Physical measurement of the contact forces applied by a collaborative robot, validated against the biomechanical limits defined in ISO/TS 15066.

Why learn:

Manufacturers publish rated forces. In practice, with tools and attached parts, the actual force can exceed the limits. In-situ validation is mandatory.

Key concepts:

ISO/TS 15066, limits by body region, transient vs. quasi-static contact, measurement devices (CBSF), recalibration frequency.

What it is:

The 4 collaborative operation modes defined by ISO 10218-2: Safety-rated Monitored Stop, Hand Guiding, Speed and Separation Monitoring (SSM), Power and Force Limiting (PFL).

Why learn:

Each application requires a specific mode. Using the wrong mode is as dangerous as having no protection at all. PFL does not mean "safe in every situation."

Key concepts:

4 ISO modes, when to use each one, combining modes, industrial vs. collaborative mode, transitions between modes.

What it is:

Adaptation of the classic Lock-Out Tag-Out procedure for robotic cells, accounting for stored energy, gravity on vertical axes, and the controller’s safety modes.

Why learn:

Robots have gravitational potential energy in their axes. Cutting the power does not mean the arm will not fall. LOTO for robots requires additional procedures that traditional LOTO does not cover.

Key concepts:

Classical vs. robotic LOTO, residual energy, mechanical brakes, zero-energy verification procedure, tag specific to teaching mode.

What it is:

A standardized sequence of actions when unplanned contact occurs between a robot and a worker: emergency stop, assistance, evidence preservation, notification.

Why learn:

The first 60 seconds after an incident determine everything: injury severity, preservation of evidence for investigation, and legal liability.

Key concepts:

Response chain (E-stop, assist, isolate, document), log preservation, mandatory notification (CIPA/SESMT), root cause investigation.

What it is:

A no-blame near-miss reporting system, including simulation of erratic human behavior (Springer 2024) as a robustness testing tool.

Why learn:

For every serious accident, there are ~300 unreported near misses (Heinrich's pyramid). Capturing near misses prevents accidents before they happen.

Key concepts:

Heinrich's pyramid, anonymous reporting, near-miss classification, simulation of erratic human behavior, trend analysis.

🔧In practice

  • ✓Complete the documented risk assessment (ISO 12100/10218) before turning on the robot
  • ✓Measure contact force in situ and validate the 4 collaborative modes (SSM/PFL)
  • ✓Test the e-stop and apply adapted LOTO, accounting for the gravitational energy of the axes
  • ✓Record every near miss without punishment—for every accident, there are ~300 near misses
View Full Version
2.3 ~30 min

🥽 Simulators, VR, and Digital Twins for Training

Market: USD 2.1B to 28.7B. VR reduces execution time by 38%. Platforms, physiology, and emergency scenarios.

What it is:

A digital twin is a virtual replica of a physical asset that synchronizes data in real time. In automotive, the market is growing from USD 2.1 billion to 28.7 billion by 2030 (CAGR 30%).

Why learn:

Digital twin training lets you make mistakes without physical consequences. Professionals can practice rare scenarios (catastrophic failure, multiple robots) that they would never encounter in practice until it was too late.

Key concepts:

Digital twin vs. simulation, two-way synchronization, acceptable latency (<100ms), model fidelity, virtual training ROI.

What it is:

Research shows that immersive VR training reduces the time needed to perform robotic tasks by up to 38% compared with conventional classroom training.

Why learn:

Each hour of VR training is equivalent to ~2.5 hours of in-person training in terms of retention and application. The investment pays for itself in months, not years.

Key concepts:

Motor learning, haptic feedback, VR-to-real skill transfer, learning curve, VR evaluation metrics.

What it is:

Virginia Tech study (2025) demonstrating the integration of digital twins with VR for industrial robot operator training, with effectiveness measured through biometrics.

Why learn:

Leading academic reference validating the hybrid training model (digital twin + VR + biometrics). Directly applicable to corporate programs.

Key concepts:

Twin-VR integration, biometric effectiveness metrics, evaluation protocol, transfer to a real-world environment, cost vs. benefit.

What it is:

Use physiological sensors during VR training to measure stress (heart rate variability), attention (eye tracking), posture, and fatigue in real time.

Why learn:

Physiological data reveals what the trainee can’t put into words: hidden stress, distraction, cognitive fatigue. It’s objective, not subjective, feedback.

Key concepts:

HRV (heart rate variability), eye-tracking (fixation vs. saccade), GSR (galvanic skin response), posture analysis, cybersickness limits.

What it is:

VR simulation of evacuation scenarios in environments with active robots: fire in a robotic cell, communication failure, robot blocking an escape route.

Why learn:

No one trains for evacuations with real robots in motion. VR is the only safe way to practice scenarios where robots interact with panicked humans.

Key concepts:

Escape routes vs. robotic zones, E-stop accessibility, emergency communication, area shutdown, cascade scenarios.

What it is:

Commercial tools for creating and operating digital twins and VR training: Siemens Xcelerator, PTC Vuforia, Scope AR (AR instructions), RoboDK (robot simulation).

Why learn:

Know which platform to use for each case: Siemens for complete lines, PTC for AR maintenance, RoboDK for offline programming, Scope AR for step-by-step instructions.

Key concepts:

Siemens Xcelerator (enterprise), PTC Vuforia (AR+IoT), Scope AR (step-by-step), RoboDK (offline programming), selection criteria, cost per license.

🔧In practice

  • ✓Practice rare scenarios (catastrophic failure, multiple robots) in the digital twin before encountering them on the production line
  • ✓Practice emergency evacuation in VR — robot blocking the route, communication failure
  • ✓Use physiological data (HRV, eye tracking) as objective feedback on stress and attention
  • ✓Choose the platform based on the use case: Siemens (production line), RoboDK (offline), Scope AR (step-by-step)
View Full Version
2.4 ~30 min

🏥 Healthcare: Nurses and Hospital Teams with Robots

Market: USD 16B to 45+B. Hospital AMRs; 78% of nurses report improved workflow. HARMONY multi-robot coordination.

What it is:

The healthcare robotics market grows from USD 16 billion to more than 45 billion by 2030, driven by a shortage of professionals and demand for efficiency.

Why learn:

Healthcare professionals who understand hospital robots will have a competitive advantage in a market that will triple in 5 years.

Key concepts:

Segments (surgical, logistics, rehabilitation, disinfection), growth drivers, adoption barriers, FDA/ANVISA regulations.

What it is:

Autonomous mobile robots that transport medications, sterile instruments, and hospital waste, operating 24/7 in corridors shared with staff and patients.

Why learn:

Nurses spend up to 30% of their shift on logistical tasks. AMRs free up that time for direct patient care.

Key concepts:

TUG/Moxi/Aethon, elevator integration, sanitation protocol, cargo traceability, navigation in dynamic environments.

What it is:

Research indicates that 78% of nurses who work with AMRs report improved daily workflow, with reduced physical fatigue and more time with patients.

Why learn:

Acceptance isn’t automatic. The 22% who are dissatisfied point to real problems: a confusing interface, lack of training, and a robot that "gets in the way" during emergencies. Understanding both sides is critical.

Key concepts:

Internal NPS, satisfaction factors, rejection factors, adoption curve by profile (early adopter vs. resistant), feedback loops.

What it is:

Practical skills hospital teams need: understanding robot navigation, how to call/cancel deliveries, and when NOT to trust the system (e.g., a rush shipment that can’t wait in the queue).

Why learn:

Most problems with hospital robots are not technical; they are about how they are used. Poorly trained staff underuse or misuse the system.

Key concepts:

Call interface, delivery prioritization, manual bypass, first-level maintenance, escalation to technical support.

What it is:

Research published in Science Robotics (2023) on robots that adjust their trajectory and speed based on urgency signals in a hospital environment (alarms, staff movement).

Why learn:

Robots that “understand” urgency reduce conflicts with the team. An AMR that stops in the hallway during a code blue is a problem. One that detours and waits is an ally.

Key concepts:

Context-aware navigation, social cues, dynamic prioritization, integration with the hospital information system (HIS), machine learning of urgency patterns.

What it is:

European HARMONY project for coordinating multiple robots in a hospital environment, and response protocol when a critical delivery (blood, organ) fails in transit.

Why learn:

Multi-robot hospital operations require orchestration. If two AMRs collide in the hallway while carrying critical loads, who has priority? The protocol defines this before it happens.

Key concepts:

HARMONY (EU H2020), hospital fleet management, load prioritization, manual fallback, critical delivery failure protocol, chain of custody.

🔧In practice

  • ✓Learn how to request, cancel, and prioritize AMR deliveries through the interface before your shift
  • ✓Know the manual bypass for an urgent load that can’t wait in the queue (e.g., code blue)
  • ✓Follow the sterilization and sterile-load traceability protocol at every cycle
  • ✓Learn about the failed critical delivery protocol (blood/organ) and escalation for support
View Full Version
2.5 ~30 min

📦 Logistics: Autonomous Warehouses and AMR Fleets

Amazon with 1M robots and DeepFleet AI. USD 1.2bi in reskilling. New roles and fleet operations KPIs.

What it is:

Amazon operates more than 1 million robots in its distribution centers, coordinated by DeepFleet AI, an artificial intelligence system that optimizes routes and allocation in real time.

Why learn:

Amazon sets the industry standard. What it does today, the rest of the market will do in 3-5 years. Understanding DeepFleet means understanding the future of logistics.

Key concepts:

DeepFleet AI, Robin/Sparrow/Proteus, multi-robot coordination, real-time route optimization, fleet scaling.

What it is:

Transformation of the warehouse role: from manual pick-pack to robotic fleet supervision, exception resolution, and kitting complex materials.

Why learn:

Jobs don’t disappear; their nature changes. Those who understand the transformation reposition themselves. Those who don’t fall behind.

Key concepts:

Manual pick-pack-ship, fleet supervision, exception handling, kitting, human-added value vs. automation.

What it is:

Amazon's USD 1.2 billion program to retrain employees: Technical Academy (9 months to become a maintenance technician) and Associate2Tech (90 days for mid-level technical roles).

Why learn:

It is the world's largest corporate robotics reskilling program. Amazon's models and metrics serve as a benchmark for any company that wants to do the same.

Key concepts:

Technical Academy (9mo), Associate2Tech (90d), conversion metrics, cost per requalified worker, post-program retention, partnership with community colleges.

What it is:

Roles that didn't exist 5 years ago: Robot Fleet Coordinator (manages a fleet of AMRs), AMR Maintenance Tech (specialized maintenance), Exception Handler (resolves problems robots can't).

Why learn:

These are careers with salaries 30-60% higher than traditional warehouse roles and growing demand. The supply of qualified professionals isn’t keeping up.

Key concepts:

Robot Fleet Coordinator, AMR Maintenance Tech, Exception Handler, AI Training Associate, salary range, recommended certifications.

What it is:

Key metrics for operating robotic fleets: System Throughput (units/hour), Exception Rate (% of tasks requiring human intervention), MTTR (Mean Time to Resolve).

Why learn:

You can't manage what you don't measure. These KPIs show whether automation is working or creating more problems than it solves.

Key concepts:

System Throughput, Exception Rate, MTTR, robotic OEE, fleet dashboards, industry benchmarks.

What it is:

Digit, Agility Robotics' humanoid robot, has been operating in GXO Logistics warehouses since 2024. It is bipedal, handles boxes, and navigates environments designed for people.

Why learn:

Humanoids in warehouses are the next frontier. Unlike AMRs, they operate in existing infrastructure without modification. This changes the skills workers need.

Key concepts:

Digit (biped), GXO deployment, box handling, human-scale navigation, humanoid vs. AMR comparison, workforce implications.

🔧In practice

  • ✓Track fleet KPIs: Throughput, Exception Rate, and MTTR on the dashboard
  • ✓Master exception handling—you solve what the robot can't
  • ✓Map the career path (Fleet Coordinator, AMR Maintenance Tech) and certifications
  • ✓Prepare for humanoids (Digit/GXO) operating in human infrastructure without modification
View Full Version
2.6 ~30 min

🌾 Agriculture, Safety, and Other Verticals

Agriculture USD 14.7bi to 48bi. 2nd-generation autonomous John Deere. Knightscope K5, Spot, Figure 02 at BMW. LGPD and human-robot coordination in a crisis.

What it is:

The agricultural robotics market is growing from USD 14.7 billion to 48 billion. John Deere leads with second-generation autonomous tractors, spraying drones, and AI-powered sensing.

Why learn:

Agriculture has the biggest labor gap (average age 58 in the U.S.). Robotics isn’t optional; it’s essential to keeping food production going.

Key concepts:

John Deere autonomous, drones + tractors + sensors, precision agriculture, age gap, startup ecosystem (Blue River, Bear Flag).

What it is:

Autonomous patrol robots: Knightscope K5 (shopping mall/campus patrol) and Boston Dynamics Spot (industrial inspection, perimeter security). Two models, two paradigms.

Why learn:

Property security with robots is real and growing, but the failures are just as instructive as the successes (NYPD K5 case).

Key concepts:

K5 (autonomous patrol), Spot (versatile quadruped), sensors (LiDAR, cameras, thermal), integration with a security center, operational limitations.

What it is:

NYPD removed the K5 due to an operational procedure failure (not a robot failure). Spot was shot during a police operation in Massachusetts (2024) and kept working. Two opposite cases.

Why learn:

The NYPD case shows that technology without operational protocols fails. The Spot case shows mechanical resilience. The lesson: the problem is rarely the hardware.

Key concepts:

Process failure vs. product failure, operating protocol, public acceptance, physical resilience, robotic safety ROI.

What it is:

Figure 02, Figure AI's second-generation humanoid robot, in pilot operation at the BMW factory. Performs assembly tasks requiring manual dexterity in spaces designed for humans.

Why learn:

BMW is the first automaker to integrate humanoids into production. The model will be replicated. Manufacturing workers need to know how to coexist with humanoids.

Key concepts:

Figure 02, bimanual manipulation, workspace sharing, training by demonstration, impact on assembly roles.

What it is:

Legal implications of robots that collect data: cameras, biometric sensors, worker tracking. Brazil’s LGPD and Europe’s GDPR directly affect deployment.

Why learn:

A security robot with a facial camera and no consent violates LGPD. Fines can reach 2% of revenue. Operators need to know what the robot collects and why.

Key concepts:

LGPD Art. 7 (legal basis), GDPR, consent, data minimization, DPO, image retention, anonymization, audit trail.

What it is:

Protocols for coordinating humans and robots during crisis situations: fire, natural disaster, intruder, power failure. Applicable to all verticals.

Why learn:

In an emergency, robots can help (reconnaissance, transport) or get in the way (blocking exits, continuing to operate). The protocol defines which behavior takes precedence.

Key concepts:

Emergency behavior modes (halt, retreat, assist), human-robot communication under stress, safe harbor zones, recovery sequence, lessons learned framework.

🔧In practice

  • ✓In agriculture, integrate autonomous tractors, drones, and sensors with remote supervision
  • ✓Remember that most failures are process-related, not hardware-related — define an operating protocol (NYPD K5 lesson)
  • ✓Ensure LGPD compliance: legal basis, data minimization, and retention of camera footage
  • ✓Define the emergency modes (halt/retreat/assist) and safe harbor zones before a crisis
View Full Version
2.7 ~45 min

🗣️ VLA: Vision-Language-Action Models

Foundation models that control robots through vision + natural language. RT-2, OpenVLA, π0, Helix, Gemini Robotics. The worker becomes an instructor, not a programmer.

What it is:

A foundation model class that receives camera images + a natural language instruction and directly generates motor actions (action tokens), end to end, combining perception, reasoning, and control.

Why learn:

Changes how you "program" a robot—from rigid code to spoken instructions; the worker becomes the instructor.

Key concepts:

End-to-end, action tokens, zero-shot generalization, transfer between tasks, foundation model.

What it is:

Google RT-2/RT-X (2023, web→robot), OpenVLA (2024, ~7B open-source), π0 by Physical Intelligence (2024, flow matching), Figure Helix (2025, real-time humanoid), Google Gemini Robotics (2025, reasoning+action).

Why learn:

Knowing what is available helps you avoid buying into promises; maturity and licensing (open vs. closed) vary.

Key concepts:

RT-2, OpenVLA 7B, π0, Helix, Gemini Robotics.

What it is:

Instead of using a teach pendant, the operator describes the task in natural language and makes corrections through demonstrations and feedback.

Why learn:

It is the interface of the near future on the factory floor; the new skill is "task prompting" and curating examples.

Key concepts:

Task prompting, demonstration/teleoperation, online correction, language grounding.

What it is:

VLAs learn from large datasets (Open X-Embodiment) + fine-tuning with local demonstrations; human-in-the-loop processes collect and validate data.

Why learn:

The bottleneck is quality data; that’s the work of the AI Training Associate.

Key concepts:

Open X-Embodiment, imitation learning, data curation, edge cases.

What it is:

VLAs fail outside the distribution and may confidently execute the wrong action ("motor hallucination"); they require safety layers (force limits, e-stop, supervision).

Why learn:

Blind trust is dangerous; ISO 10218 and human-in-the-loop remain mandatory.

Key concepts:

Out-of-distribution, action hallucination, safety envelope, human supervision.

What it is:

Shifts the key skill from “coding movement” to “teaching and supervising behavior”; the role of robot instructor / physical AI operator emerges.

Why learn:

Defines the reskilling curriculum.

Key concepts:

Robot instructor, Physical AI, fleet supervision with VLA, upskilling.

🔧In practice

  • ✓Write the “task prompt” in clear language and test the behavior before releasing it to production
  • ✓Collect and curate local demonstrations (teleop) for fine-tuning—data quality matters more than volume
  • ✓Map cases outside the distribution and cover them with a safety envelope (force limit + accessible e-stop)
  • ✓Keep a human in the loop: never blindly trust the action generated by the model
View Full Version
2.8 ~50 min

💻 How to Program Robots: From the Teach Pendant to Code

From the traditional approach (teach pendant) to code (Python, ROS 2) and the new frontier of programming with natural language. A pathway by profile for the reskilled worker.

What it is:

Program using a console (teach pendant) and by guiding the arm by hand (lead-through) while recording positions; the foundation of cobots (UR, FANUC CRX).

Why learn:

The dominant method today and the entry point for anyone who has never coded.

Key concepts:

Waypoints, pose, base/tool frames, speed/acceleration, manual jog.

What it is:

Visual interfaces (Blockly-style blocks, flowcharts, UR PolyScope drag-and-drop) build logic without text.

Why learn:

Lets operators without a technical background automate simple tasks; democratizes access.

Key concepts:

if/loop/wait blocks, action nodes, sensor-based conditionals, templates.

What it is:

The most widely used language in robotics/AI; controls cobots via SDK (e.g., ur_rtde), processes vision, and integrates AI.

Why learn:

Moving beyond no-code opens the door to complex tasks and better pay.

Key concepts:

Manufacturer SDK, control loop, sensor readings, vision integration, reusable scripts.

from rtde_control import RTDEControlInterface as RTDEControl
robo = RTDEControl("192.168.0.10")
pose = [-0.14, -0.40, 0.20, 0, 3.14, 0]  # x, y, z + rotacao
robo.moveL(pose, speed=0.25, acceleration=0.5)
robo.setStandardDigitalOut(0, True)  # fecha a garra
What it is:

ROS 2 is the standard middleware (it isn’t a real OS): nodes, topics (publish/subscribe), messages, services; Nav2, MoveIt 2, Gazebo.

Why learn:

AMRs, humanoids, and research run ROS 2; it is a prerequisite for technical roles.

Key concepts:

Nodes, topics, messages, Nav2, MoveIt 2, simulation.

What it is:

Describe the task in Portuguese and the robot generates/executes it (connects to 2.7 VLA); “code-as-policies” = an LLM writes the code, a human reviews it.

Why learn:

Industry direction; the skill becomes specifying the task well and reviewing what AI generated.

Key concepts:

Code-as-policies, task prompt, VLA, human review, safety of self-generated code.

What it is:

Operator (teach pendant→no-code), technician (no-code→Python), engineer (Python→ROS 2→VLA).

Why learn:

Knowing the sequence prevents skipping steps; each level opens up a salary tier and connects to certifications (module 3.4).

Key concepts:

Track by role, from simple to complex, project portfolio, continuous learning.

🔧In practice

  • ✓Start by recording waypoints on the teach pendant and validate poses, speed, and base/tool frames
  • ✓Build your first no-code routine (PolyScope/blocks) with if/loop and a sensor-based conditional
  • ✓Rewrite this routine in Python via the SDK (e.g., ur_rtde) to gain fine-grained control and reuse
  • ✓Follow the path for your role (operator→technician→engineer) and always review code generated by an LLM/VLA
View Full Version
Track 1 - Fundamentals Track 3 - Strategic