🔍 Detect objects with YOLO26

A object detector takes an image and returns a list: for each thing it recognizes, a box (where it is), a class (what it is), and a confidence score (from 0 to 1, how sure the model is). YOLO (“you only look once”) does this by looking at the image just once, which makes it fast enough for robots. Ultralytics’ pretrained models already recognize the 80 classes in the COCO dataset, such as person, car, dog, bottle, and chair.

How to read this map: each colored box is a detected object, with a “class confidence” label. The purple panel on the right shows what reaches your program for the person: numbers only. The bottle has lower confidence (0,67) because it is small and partly in shadow; that’s where errors arise (topic 6).

Why learn this

The color mask fails with objects that have common colors or when the lighting changes. The detector recognizes shape, so it can follow a person wearing a shirt of any color. In return, it uses much more processing power. Knowing what each output means lets you use it to control the robot.

Key concepts

O YOLO26 is the current generation of the Ultralytics YOLO family, the successor to YOLO11. Tutorials using older versions are still around, but today’s approach is the ultralytics with the yolo26n.ptweights n, s, m, l e x. It comes in n: each step up is a little more accurate and runs much more slowly. On the Pi 5, always start with the

How to read this map: Each step is a YOLO26 size. The highlighted "n" is the only one that keeps the robot responsive in real time using just the Pi's processor. The larger ones are useful for training comparisons on a PC or running with an accelerator.

Goal: each step is a YOLO26 size. The highlighted “n” is the only one that keeps the robot responding in time using just the Pi’s processor. Larger ones are useful for training comparisons on a PC or running with an accelerator.

source ~/robo-venv/bin/activate
pip install ultralytics                        # also installs PyTorch for ARM
yolo predict model=yolo26n.pt source="https://ultralytics.com/images/bus.jpg"   # quick test

# YOLO26 live on the Camera Module 3: prints what it sees and the time for each frame
import sys, time
from picamera2 import Picamera2
from ultralytics import YOLO

MODELO = sys.argv[1] if len(sys.argv) > 1 else "yolo26n.pt"   # switch to NCNN in topic 3
modelo = YOLO(MODELO)

picam2 = Picamera2()
picam2.configure(picam2.create_preview_configuration(main={"size": (640, 480), "format": "RGB888"}))
picam2.start(); time.sleep(1)
print(f"Model {MODELO} loaded. Press Ctrl+C to exit.")
try:
    while True:
        quadro = picam2.capture_array()                 # BGR, as expected by Ultralytics
        t0 = time.perf_counter()
        r = modelo(quadro, imgsz=640, conf=0.5, verbose=False)[0]
        ms = (time.perf_counter() - t0) * 1000
        achados = [f"{r.names[int(c)]} {float(p):.2f}" for c, p in zip(r.boxes.cls, r.boxes.conf)]
        print(f"{ms:6.0f} ms | {1000 / ms:4.1f} FPS | {', '.join(achados) or 'nothing'}")
except KeyboardInterrupt:
    picam2.stop()

Why learn this

With about twenty lines, you have a robot that recognizes people and everyday objects. This same loop (capture, infer, read the boxes) is the basis for project 3.6 and appears again in Track 5, when the arm needs to find an object before picking it up.

Key concepts

The .pt format is great for training, but it isn’t the fastest for running on an ARM processor. Ultralytics exports the same model to other formats; according to the official Raspberry Pi guide, NCNN is fastest on the Pi’s CPU. The boxes and classes stay the same; only the “engine” doing the calculations changes. The other way to improve speed is to change the input size: imgsz=320 processes a quarter of the pixels of 640, but loses accuracy on small objects.

How to read this map: only the top bar has a reference number (from the Ultralytics guide for the Pi 5). The others show the direction of the improvement; find the exact time by running the script below on your Pi, with your cooler and camera.

Goal: export yolo26n to NCNN and compare the two formats on the same image, with numbers.

cd ~/robo && source ~/robo-venv/bin/activate
yolo export model=yolo26n.pt format=ncnn       # creates the yolo26n_ncnn_model/ folder
python yolo_camera.py yolo26n_ncnn_model       # the same script from topic 2, now using NCNN

# Compares PyTorch vs. NCNN (640 and 320) on the same image: average of 20 inferences
import time
from ultralytics import YOLO

IMAGEM = "https://ultralytics.com/images/bus.jpg"   # or the path to one of your photos
testes = [("yolo26n.pt", 640), ("yolo26n_ncnn_model", 640), ("yolo26n_ncnn_model", 320)]

for caminho, tamanho in testes:
    modelo = YOLO(caminho, task="detect")
    for _ in range(3):                              # warm-up: not counted
        modelo(IMAGEM, imgsz=tamanho, verbose=False)
    tempos = []
    for _ in range(20):
        r = modelo(IMAGEM, imgsz=tamanho, verbose=False)[0]
        tempos.append(r.speed["inference"])           # ms, network only
    media = sum(tempos) / len(tempos)
    print(f"{caminho:22s} imgsz={tamanho}: {media:6.1f} ms  ({1000 / media:4.1f} FPS)  objects={len(r.boxes)}")

Why learn this

Doubling the FPS at no extra cost is the difference between a robot that follows a person and one that loses them at every turn. The habit of measuring before and after applies to any optimization you make from here on.

Key concepts

When NCNN and imgsz aren’t enough, the next step is to move the neural network off the processor. The AI HAT+ is a board that fits on top of the Pi 5 and connects through the PCIeconnector; it has a Hailo chip designed specifically for inference. There are two generations: AI HAT+ with Hailo-8L (13 TOPS) or Hailo-8 (26 TOPS), designed for vision, and AI HAT+ 2 with Hailo-10H (40 TOPS in INT4) and 8 GB of its own memory, which can also run small language and vision-language models. Both work only with the Pi 5; the old "AI Kit" has been discontinued.

How to read this map: On the left, the CPU handles all three tasks, and the middle one (red) dominates. On the right, the heavy processing runs on the NPU over PCIe, and only the result comes back. The CPU stays free for the rest of the robot: video server, communication with the ESP32, logs.

Step by step (overview)

lspci | grep -i hailo        # Does the board show up on the PCIe bus?
vcgencmd get_throttled       # 0x0: the power supply can handle the Pi + HAT + camera setup

Why learn this

An accelerator isn’t required for this track: project 3.6 works with CPU and NCNN. But knowing when it’s worthwhile (many objects, high FPS, language AI on the same robot) helps you avoid buying one too early or pushing the CPU for too long.

Key concepts

COCO doesn’t include "my mug," "orange track cone," or "competition part." For that, you do a fine-tune: start with yolo26n, which already knows how to recognize shapes, and show it a few hundred of your photos with the boxes marked. Training runs on a PC with an NVIDIA GPU or in Google Colab (free GPU in your browser); the Pi only receives the finished result. Training on the Pi can work, but it’s far too slow to be worthwhile.

How to read this map: The purple steps happen outside the robot; the cyan step is the robot using the model. The dashed loop is the key: the first model always fails in some cases, and each error you photograph and label improves the next training run.

Step by step

Goal: describe the dataset and train a two-class detector starting from yolo26n. Replace the path and class names with yours.

# Expected structure:
#   my_dataset/images/train/*.jpg   my_dataset/labels/train/*.txt
#   my_dataset/images/val/*.jpg     my_dataset/labels/val/*.txt
path: /caminho/para/meu_dataset
train: images/train
val: images/val
names:
  0: caneca
  1: cone

pip install ultralytics
yolo detect train data=meu_dataset/data.yaml model=yolo26n.pt epochs=100 imgsz=640 batch=16
# at the end: runs/detect/train/weights/best.pt  (the best model on validation)
yolo detect val model=runs/detect/train/weights/best.pt data=meu_dataset/data.yaml

# then, on the Pi:
# scp best.pt <your_username>@robo.local:~/robo/
# yolo export model=best.pt format=ncnn   →   python yolo_camera.py best_ncnn_model

Why learn this

This is where the robot's vision starts serving your project, instead of the other way around: competitions, sorting parts, a robot that searches for a specific object. And the process (collect, label, train, test, repeat) is the same one you use in Track 5 to teach the arm through demonstrations.

Key concepts

A detector doesn't "see" the way you do: it makes a statistical guess for every frame. In low light, the object looks dark and blurry; far away, it takes up few pixels after resizing; and sometimes a backpack becomes a "person" for one frame. The confidence threshold is the control you can adjust: too high and the robot misses the target; too low and it chases ghosts. That's why the robot's code never reacts to just one frame.

How to read this map: the scale shows the value of conf. On the left (red), the robot reacts to everything; on the right (cyan), only to very clear objects. The purple frame is a reasonable starting point, and it becomes much more stable when you require the object to appear in a few consecutive frames.

Goal: turn raw detections into a stable decision: only the class that matters, only boxes large enough, and only after 3 consecutive frames. The result is the horizontal error, just like in module 3.2.

# Stable decision from YOLO26: class, minimum size, and persistence
import time
from picamera2 import Picamera2
from ultralytics import YOLO

ALVO = "person"          # class name in the model (COCO uses English names)
CONF_MIN = 0.5
AREA_MIN = 0.02          # box covers at least 2% of the image
QUADROS_SEGUIDOS = 3
LARGURA, ALTURA = 640, 480

modelo = YOLO("yolo26n_ncnn_model", task="detect")
picam2 = Picamera2()
picam2.configure(picam2.create_preview_configuration(main={"size": (LARGURA, ALTURA), "format": "RGB888"}))
picam2.start(); time.sleep(1)
contador = 0
try:
    while True:
        r = modelo(picam2.capture_array(), imgsz=640, conf=CONF_MIN, verbose=False)[0]
        melhor = None
        for (x1, y1, x2, y2), c in zip(r.boxes.xyxy.tolist(), r.boxes.cls.tolist()):
            area = (x2 - x1) * (y2 - y1) / (LARGURA * ALTURA)
            if r.names[int(c)] == ALVO and area >= AREA_MIN:
                if melhor is None or area > melhor[1]:
                    melhor = ((x1 + x2) / 2, area)          # center x and size
        contador = contador + 1 if melhor else 0
        if contador >= QUADROS_SEGUIDOS:
            cx, area = melhor
            print(f"STABLE TARGET  error={cx - LARGURA / 2:+5.0f}  size={area:.2f}")
        else:
            print(f"no stable target ({contador}/{QUADROS_SEGUIDOS}) → robot stopped")
except KeyboardInterrupt:
    picam2.stop()

Why learn this

A person-following robot is the track's project, and it's only safe if you know where vision fails. Threshold, minimum size, persistence, and stopping when the target is lost are the four safeguards that separate a nice-looking demo from a robot you can trust.

Key concepts