>_ ctrlaltbrian
raspberry-pi intermediate 45 min

Raspberry Pi: gesture recognition with MediaPipe

Run hand gesture recognition on a Raspberry Pi camera feed with MediaPipe: count fingers, detect an open palm or fist, and trigger real actions in Python.

Code available for: Python
Published Sep 22, 2026

I wanted to wave at a camera and have something happen. Lights on, a photo taken, a relay clicked. The old way to do that is OpenCV with color masks and contour math, and the results were fine until the lighting changed or somebody wore a red shirt. MediaPipe is the newer answer: Google’s hand-tracking model runs on-device, finds 21 landmarks per hand in every frame, and you write plain Python against the landmarks. No training, no dataset, no GPU.

The trap is expectations on Pi hardware. MediaPipe’s full hand model is comfortable at 20 to 30 fps on a Pi 5, roughly 10 to 15 fps on a Pi 4, and the demo videos you have seen are not shot on a Zero 2 W. Plan for gestures you hold for half a second, not flicks, and the Pi hardware is genuinely enough. (e.g. an open palm held up for a moment to turn the lamp on.) If you want the color-mask approach for comparison, the OpenCV camera tutorial covers that side of the house.

What you need

Needed

  • Raspberry Pi 4 or Pi 5 with Raspberry Pi OS Bookworm (64-bit). The
  • Camera Module v3 or a USB webcam, working in Picamera2 or OpenCV.
  • A screen or VNC session for development. You can run the final
  • About 45 minutes

Nice to have

  • An anti-static wristband (cheap insurance around the Pi’s GPIO header)
  • A small screwdriver kit for the case and camera ribbon clips

Wiring

No GPIO in this one. The camera ribbon goes into the CSI connector with the contacts facing the heat sink on a Pi 4 (toward the lens side on a v2/v3 camera), latch pressed down. That is the whole hardware project. If the camera is not detected, the fix is usually reseating the ribbon or enabling it: Raspberry Pi Configuration >> Interfaces >> Camera >> Enabled.

Install

Everything comes from apt and pip, in this order, because the system OpenCV and NumPy have to exist before MediaPipe’s wheel settles in:

sudo apt update
sudo apt install -y python3-opencv python3-picamera2
python3 -m venv ~/mpenv --system-site-packages
source ~/mpenv/bin/activate
pip install mediapipe-rpi4 mediapipe

(mediapipe-rpi4 is the community wheel built for Pi boards; if the plain mediapipe wheel installs on your Pi, use it, it is the newer build. Either one gives you the same mp.solutions.hands API.)

The venv with --system-site-packages is not optional flavor text. Picamera2 lives in the system Python, and a bare venv cannot see it, which produces the exact “no module named picamera2” error that wastes an evening.

The code

This script opens the camera, finds hands, counts extended fingers, and reacts to three gestures: open palm, fist, and victory. Swap the do_action bodies for whatever your project needs.

#!/usr/bin/env python3
"""Gesture recognition with MediaPipe on a Raspberry Pi.

Gestures: OPEN_PALM, FIST, VICTORY. Prints and reacts when a gesture
is held for HOLD_FRAMES consecutive frames (debounce on purpose).
"""
import time
import cv2
import mediapipe as mp
from picamera2 import Picamera2

# The 21 landmarks per hand have fixed indices. These four sets are
# the tips and the two joints of each finger, plus the wrist.
FINGER_TIPS = [4, 8, 12, 16, 20]   # thumb, index, middle, ring, pinky
FINGER_PIPS = [3, 6, 10, 14, 18]   # one joint below each tip
WRIST = 0

HOLD_FRAMES = 5          # consecutive frames a gesture must persist
COOLDOWN = 1.5           # seconds between fired actions

mp_hands = mp.solutions.hands
hands = mp_hands.Hands(
    max_num_hands=1,
    model_complexity=0,   # 0 = lighter model, faster on a Pi
    min_detection_confidence=0.6,
    min_tracking_confidence=0.5,
)

picam = Picamera2()
picam.configure(picam.create_preview_configuration(
    main={"format": "RGB888", "size": (640, 480)}))
picam.start()
time.sleep(1.0)

last_action = ""
cooldown_until = 0.0


def count_extended_fingers(landmarks, handedness):
    """Count fingers whose tip is farther from the wrist than its joint."""
    wrist = landmarks[WRIST]
    count = 0
    # Thumb: compare x, because the thumb extends sideways.
    thumb_tip = landmarks[FINGER_TIPS[0]]
    thumb_pip = landmarks[FINGER_PIPS[0]]
    if handedness == "Left":
        if thumb_tip.x > thumb_pip.x:
            count += 1
    else:
        if thumb_tip.x < thumb_pip.x:
            count += 1
    # Other four fingers: tip above (smaller y) than the joint = extended.
    for tip, pip in zip(FINGER_TIPS[1:], FINGER_PIPS[1:]):
        if landmarks[tip].y < landmarks[pip].y:
            count += 1
    return count, wrist


def do_action(gesture):
    """Put your real actions here."""
    if gesture == "OPEN_PALM":
        print("ACTION: lights on")     # (e.g. GPIO.output(relay, HIGH))
    elif gesture == "FIST":
        print("ACTION: lights off")
    elif gesture == "VICTORY":
        print("ACTION: take a photo")


while True:
    frame = picam.capture_array()
    results = hands.process(frame)

    gesture = ""
    if results.multi_hand_landmarks:
        lm = results.multi_hand_landmarks[0]
        handedness = results.multi_handedness[0].classification[0].label
        n, wrist = count_extended_fingers(lm.landmark, handedness)
        if n >= 4:
            gesture = "OPEN_PALM"
        elif n <= 1:
            gesture = "FIST"
        elif n == 2:
            # Two extended neighbors (index + middle) = victory.
            gesture = "VICTORY"
        # Draw the skeleton so you can see what the model sees.
        mp.solutions.drawing_utils.draw_landmarks(
            frame, lm, mp_hands.HAND_CONNECTIONS)
        cv2.putText(frame, f"{gesture or '...'} fingers={n}",
                    (10, 40), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)

    now = time.time()
    if gesture and gesture == last_action and now > cooldown_until:
        held_frames = getattr(do_action, "held", 0) + 1
        do_action.held = held_frames
        if held_frames >= HOLD_FRAMES:
            do_action(gesture)
            cooldown_until = now + COOLDOWN
            do_action.held = 0
    else:
        do_action.held = 0
    last_action = gesture

    cv2.imshow("gestures", frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

picam.stop()
cv2.destroyAllWindows()

Run it with the venv active: python gestures.py. Raise an open palm and watch the label settle, then close it into a fist and watch the fingers counter drop. If the overlay draws and the counters do not move, your hand is too far from the camera: the model wants a hand roughly half a meter away, not across the room.

One honest note about MediaPipe’s maintenance: the pip package has been in maintenance mode while Google works on its successor. It still installs and runs fine, but pin the version that works on your Pi and keep a copy of the wheel. That is standard practice for any model dependency you cannot rebuild yourself.

What you learned

  • MediaPipe turns gesture recognition into landmark arithmetic: the model finds 21 points per hand, and your Python is just comparisons on those points.
  • Debounce matters more than accuracy. Requiring HOLD_FRAMES consecutive frames plus a cooldown turns a jittery 85% accurate classifier into one that does not flicker the lights.
  • The pattern in 1 sentence: camera frame in, landmarks out, compare landmarks, act on a held gesture.

When something breaks

  • “No module named mediapipe”. You are on 32-bit Raspberry Pi OS, or the pip install went to the wrong interpreter. Check getconf LONG_BIT (want 64) and install inside the venv you actually run.
  • “No module named picamera2” inside the venv. You created the venv without --system-site-packages. Delete it and redo the install block, in order.
  • Detection misses my hand entirely. Lighting first (backlit windows are the enemy), then distance (half a meter), then confidence: drop min_detection_confidence to 0.5 and see if that is the difference between usable and not.
  • The thumb count is wrong when I rotate my hand. The thumb comparison is x-axis based and it assumes an upright hand. Rotate less or add a y-axis check; the four fingers are more reliable than the thumb either way.
  • fps is under 5 on a Pi 4. Lower the camera to 480p (that is what the config above already does), keep model_complexity=0, and close the desktop browser you left open. A headless run with no imshow gains another couple of frames if you only need the actions.

What to build next

Swap do_action for an HTTP call to the Flask API tutorial and the gesture becomes a trigger for any sensor dashboard you already have. Log every fired gesture to SQLite with the logging tutorial and you get a history of what the camera actually saw. For the natural escalation, pair this with the MJPEG stream tutorial: the same camera shows a live browser view while the gesture script runs in parallel, so you can watch your own hand trigger the system. —Brian