GestureAI
A human-computer interaction system that uses computer vision and machine learning to play rock-paper-scissors by detecting hand gestures in real time through a webcam.
🧠 The idea
GestureAI was built as a project for a Big Data and Artificial Intelligence course: turning a webcam into a game controller by detecting hand gestures in real time to play rock-paper-scissors against the machine.
🚀 What it does
- Detects hand gestures in real time from the webcam
- Classifies the gesture (rock, paper, or scissors) using a trained model
- Resolves the round against a randomized AI move
- Logs results and persists match metrics in SQLite
- Streamlit dashboard for analyzing the user experience
👥 Team project
Built together with three teammates, each responsible for a different layer of the system. My role was ML Engineer: training, validating, and optimizing the gesture classification model, based on hand landmarks extracted with MediaPipe.
🏗️ Architecture
The system runs as a sequential pipeline:
- Capture — real-time video via OpenCV
- Processing — extraction of 21 hand landmarks with MediaPipe
- Inference — gesture classification via the trained model
- Logic — resolving the round against the AI’s move
- Output — results display and persistence in SQLite
📊 What made it interesting
The hard part wasn’t the game itself, it was:
- Making gesture classification reliable across different lighting conditions and camera angles
- Choosing and tuning the model so inference ran in real time with no noticeable lag
- Coordinating four pieces of the system (data, ML, game engine, BI) built in parallel by different people
🎯 Current status
Delivered as part of the AI and Big Data specialization course. Fully functional end-to-end: capture, classification, gameplay, and metrics dashboard.