Control games and applications using hand gestures detected through your webcam
Features • Installation • Usage • How It Works • Configuration
visision.demo.-.Made.with.Clipchamp.2.mp4
|
Intuitive hand gesture controls for steering, acceleration, and special actions |
Smooth 30+ FPS hand tracking with minimal latency using MediaPipe |
Adjustable thresholds, sensitivity settings, and visual feedback |
- 🎯 Precise Steering Control - Tilt your hands like a steering wheel
- 📏 Distance Detection - Hands close/far triggers different actions
- 👆 Finger Gestures - Forward/backward with finger combinations
- 🎨 Visual Feedback - Real-time overlay showing detected gestures
- 🔧 Easy Calibration - Auto-centering and manual threshold adjustment
- 💾 Persistent Settings - Your preferences are saved between sessions
| Component | Minimum | Recommended |
|---|---|---|
| OS | Windows 10 / macOS 10.14 / Ubuntu 18.04 | Windows 11 / macOS 12+ / Ubuntu 22.04 |
| Python | 3.8 | 3.10+ |
| RAM | 4 GB | 8 GB+ |
| Webcam | 720p @ 30fps | 1080p @ 30fps |
| CPU | Dual-core 2.0 GHz | Quad-core 2.5 GHz+ |
opencv-python>=4.5.0
mediapipe>=0.10.0
numpy>=1.21.0
# Clone the repository
git clone https://github.com/YOUR_USERNAME/gesture-control-game.git
cd gesture-control-game
# Create virtual environment (recommended)
python -m venv venv
# Activate virtual environment
# On Windows:
venv\Scripts\activate
# On macOS/Linux:
source venv/bin/activate
# Install dependencies
pip install -r requirements.txtpip install opencv-python mediapipe numpy🐧 Linux Additional Steps
On Linux, you may need to install additional packages:
sudo apt-get update
sudo apt-get install -y libgl1-mesa-glx libglib2.0-0🍎 macOS Camera Permissions
On macOS, you'll need to grant camera permissions:
- Go to System Preferences → Security & Privacy → Privacy
- Select Camera from the left sidebar
- Check the box next to your terminal application or Python
python main.pypython main.py --camera 0 # Select camera index (default: 0)
python main.py --width 1280 # Set capture width
python main.py --height 720 # Set capture height
python main.py --no-gui # Run without visual overlay| Key | Action |
|---|---|
Q / ESC |
Quit application |
R |
Reset/recalibrate |
S |
Toggle skeleton overlay |
T |
Toggle trails |
+ / - |
Adjust sensitivity |
SPACE |
Pause detection |
The steering is calculated based on the vertical difference (dy) between your left and right wrist positions:
┌─────────────────────────────────────────────────────────────┐
│ │
│ LEFT HAND HIGH NEUTRAL RIGHT HAND HIGH│
│ ┌──┐ ┌──┐ ┌──┐ ┌──┐ │
│ │ │ │ │ │ │ │ │ │
│ └──┘ ┌──┐ └──┘ └──┘ ┌──┐ └──┘ │
│ │ │ ═══════ │ │ │
│ └──┘ └──┘ │
│ │
│ STEER LEFT NO STEERING STEER RIGHT │
│ │
└─────────────────────────────────────────────────────────────┘
flowchart LR
A[📷 Webcam] --> B[🖼️ Frame Capture]
B --> C[🤖 MediaPipe]
C --> D[✋ Hand Landmarks]
D --> E[📐 Gesture Analysis]
E --> F[🎮 Game Input]
E --> G[📊 Visual Feedback]
🔄 Steering (Two Hands)
- How: Hold both hands up, tilt like a steering wheel
- Detection: Compares vertical position of left vs right wrist
- Output: Continuous value from -60° to +60°
- Threshold: Steering activates when angle exceeds ±35° (configurable)
📏 Distance (Two Hands)
- Close: Hands within ~12% of frame width → triggers "close" action
- Far: Hands beyond ~55% of frame width → triggers "far" action
- Use Case: Zoom, brake/accelerate, menu navigation
👆 Finger Gestures (Single Hand)
- Forward: Index + Middle fingers extended, others closed
- Backward: Only thumb extended
- Use Case: Acceleration, item selection, confirmation
Settings are stored in config.json:
{
"thresholds": {
"steering_angle": 35.0,
"hands_close_dist": 0.12,
"hands_far_dist": 0.55,
"finger_extend_thresh": 0.06,
"stability_delay": 0.18,
"steering_dy_scale": 180.0,
"visual_max_angle": 60.0
},
"sensitivity": {
"steering": 1.0,
"distance": 1.0,
"fingers": 1.0
},
"display": {
"show_skeleton": true,
"show_trails": true,
"show_steering_bar": true
}
}| Parameter | Default | Description |
|---|---|---|
steering_angle |
35.0 | Degrees of tilt needed to trigger steering |
hands_close_dist |
0.12 | Distance threshold for "close" detection |
hands_far_dist |
0.55 | Distance threshold for "far" detection |
stability_delay |
0.18 | Seconds a gesture must hold before triggering |
steering_dy_scale |
180.0 | Multiplier for vertical difference to angle |
gesture-control-game/
├── 📄 main.py # Application entry point
├── 📄 gesture_detector.py # Core gesture detection logic
├── 📄 utils.py # Helper utilities (smoothing, filters)
├── 📄 config.json # User configuration
├── 📄 requirements.txt # Python dependencies
├── 📄 README.md # This file
❌ Camera not detected
- Check if another application is using the camera
- Try a different camera index:
python main.py --camera 1 - Verify camera permissions (especially on macOS)
- Test camera with another app first
❌ Poor hand detection
- Ensure good lighting (avoid backlight)
- Use a plain background if possible
- Keep hands within frame and visible
- Adjust
min_detection_confidencein code (default: 0.4)
❌ Steering not centered
- Press
Rto recalibrate - Hold hands level when starting
- Adjust
steering_dy_scalein config if consistently off-center
❌ Laggy performance
- Reduce camera resolution:
--width 640 --height 480 - Close other applications
- Disable visual overlays:
--no-gui - Check CPU usage - MediaPipe is CPU-intensive
❌ MediaPipe installation fails
# Try upgrading pip first
pip install --upgrade pip
# Install with specific version
pip install mediapipe==0.10.9
# On Apple Silicon Macs
pip install mediapipe-siliconContributions are welcome! Here's how you can help:
- Fork the repository
- Create a feature branch:
git checkout -b feature/amazing-feature - Commit your changes:
git commit -m 'Add amazing feature' - Push to the branch:
git push origin feature/amazing-feature - Open a Pull Request
# Clone your fork
git clone https://github.com/YOUR_USERNAME/gesture-control-game.git
# Install dev dependencies
pip install -r requirements-dev.txt
# Run tests
python -m pytest tests/