Overview

mmScribe is an innovative Aerial Handwriting system that enables contactless human-computer interaction through millimeter-wave radar technology. The system accurately captures user handwriting and converts them into text input, providing a novel approach to human-computer interaction.

Key Features

• 🎯 Streaming Aerial Handwriting Recognition - Real-time gesture-to-text conversion
• 📱 Cross-platform Compatibility - Supports Android, Windows, and Raspberry Pi
• ⚡ Real-time Response - Low latency for seamless interaction
• 🔒 Privacy-Preserving - No camera required, protecting user privacy
• 🛠️ Easy Integration - Simple integration with existing systems
• 📊 Comprehensive Tools - Complete data analysis and processing toolkit

Platform Support

mmScribe supports multiple platforms through our runtime system:

Hardware Requirements

• ESP32-BGT60TR13 Radar Module
  • 58-63GHz mmWave Radar
  • USB/UART Interface
  • 5V Power Supply

Quick Installation

Android

# Download and install APK
wget https://github.com/Tkwer/mmScribe/releases/latest/download/mmScribe.apk

Windows/Raspberry Pi

# Clone the repository
git clone https://github.com/Tkwer/mmScribe.git
cd mmScribe

# Install dependencies
pip install -r requirements.txt

Dataset

We provide a comprehensive dataset for aerial handwriting recognition using millimeter-wave radar:

• 🧑‍🤝‍🧑 12 participants (6 males, 6 females)
• 📝 15,488 total samples across all participants
• 📊 Rich feature set including micro-Doppler and range-time data
• 🎯 Ground truth data from Leap Motion controller

Dataset Structure

dataset/
├── datas1/    # Reserved dataset
├── datas2/    # Participant 001 (1212 samples)
├── datas3/    # Participant 002 (1202 samples)
...
└── datas14/   # Participant 013 (1192 samples)

Quick Start

Prerequisites

• Python 3.8 or higher
• CUDA-compatible GPU (optional, for faster processing)
• Compatible radar hardware (ESP32-BGT60TR13)

Basic Usage

1. Select Dataset - Choose from our comprehensive dataset
2. Run Training:

   python main.py
   

3. Deploy - Use the trained model on your target platform

Technical Details

The system leverages millimeter-wave radar technology to capture fine-grained hand movements in 3D space. By analyzing micro-Doppler signatures and range-time data, mmScribe can accurately recognize air handwriting without requiring any physical contact or camera-based tracking.

Key Advantages

• Privacy-First Design: No visual data collection
• Works in Any Lighting: Independent of ambient light conditions
• Low Power Consumption: Efficient radar-based sensing
• Robust Performance: Resistant to environmental interference

Applications

• 📝 Contactless text input for smart devices
• 🏥 Sterile environment interaction (medical settings)
• 🎮 Gaming and entertainment interfaces
• 🏭 Industrial control systems
• ♿ Accessibility solutions for users with mobility challenges

Documentation

For detailed documentation, including:

• Runtime setup guides
• Dataset usage instructions
• API references
• Training tutorials

Please visit the GitHub Repository.

License

This project is licensed under the MIT License - see the LICENSE file for details.

⭐ If you find this project useful, please consider giving it a star on GitHub!