Project Overview

This project presents a gesture-controlled robotic car system that uses computer vision to translate human body movements into real-time driving commands. By combining an Android application for gesture recognition and an ESP32-based robotic vehicle, the system enables intuitive, controller-free operation with live first-person video feedback from the robot.

Demo Video & System Architecture

Architecture Architecture

The system consists of two main components: an Android application and an ESP32-based robotic car. The phone captures the user’s movements and converts them into control commands, which are sent wirelessly to the robot. At the same time, the robot streams real-time video back to the phone, forming a closed-loop interactive control system.

Key Features

Control Mode Switching for Gesture-Based Robotic Car

The system provides three control modes to adapt to indoor navigation and real-world driving behavior. Each mode defines how the car responds to recognized gestures.

Dexter overview
Dexter overview

Control Modes

  • Mode 0 (OFF)
    • All gestures are ignored.
    • Default mode when the app starts.
    • Prevents accidental movement before the user is ready.
  • Mode 1 – Limited Space Control Mode
    • Designed for indoor or confined spaces.
    • Movement and turning gestures are recognized independently.
    • Allows rotating in place without forward motion.
    • Commands:
      • move1 — move forward
      • left1 — rotate left in place
      • right1 — rotate right in place
  • Mode 2 – Real Driving Simulation Mode
    • Mimics real vehicle steering behavior.
    • Turning only works when moving forward.
    • Commands:
      • move2 — move forward
      • left2 — turn left while moving
      • right2 — turn right while moving

Mode Switching Mechanism

  • Bring both palms together and hold for a fixed time.
  • Mode order: OFF → Mode 1 → Mode 2 → OFF
  • Cooldown prevents accidental switching.

Visual Mode Feedback

  • Red — OFF
  • Green — Limited Space
  • Yellow — Real Driving

Technologies Used