Can we use simple hand gestures to enter Braille on a computer or digital device?
In this thesis, you will design and test 6 hand gestures for Braille input. You will collect gesture data from real participants using a camera, track their hand movements, and use AI and computer vision to recognize the gestures.
You will also study which gestures are fast, accurate, easy, and comfortable to use by measuring errors, input time, hand movement, two-hand coordination, and fatigue.
The goal is to build a small gesture-based Braille input system and identify which gestures work best for real users.
Hand gestures are becoming an increasingly useful way to interact with computers and digital devices.
This project explores gesture-based input as an accessible way to enter Braille. It combines AI, computer vision, and Human–Computer Interaction (HCI) while working with real human data.
The project is part of Gesture Braille, a larger research project exploring a camera-based digital text input system for people with visual impairments.
Future work can include:
Improve recognition using AI and deep learning.
Enable real-time gesture recognition.
Test with more users and real-world conditions.
Explore mobile and wearable versions.
Convert gestures into digital text or speech.
Develop the complete Gesture Braille system and publish the research.
Python and AI/ML
Computer vision and hand tracking
Dataset collection and analysis
Gesture classification
HCI and user studies
Research and technical writing
Artificial Intelligence
Machine Learning
Computer Vision
Human–Computer Interaction
Statistics / Probability
Programming
BrailleSketch: A Gesture-based Text Input Method for People with Visual Impairments — ASSETS 2017
Useful for understanding gesture-based Braille input and user evaluation.
Hybrid-Brailler: Combining Physical and Gestural Interaction for Mobile Braille Input and Editing — CHI 2018
Useful for understanding different approaches to digital Braille input.
Vision-Based Air-Writing for Mobile Braille Input
Useful for understanding camera-based hand tracking and AI-based Braille input.
From One to Many Users and Contexts: A Classifier for Hand and Arm Gestures — IUI 2015
Useful for understanding gesture classification across different users and situations.
An Interactive Braille Recognition System for the Visually Impaired Based on a Portable Camera — CHI 2005
Useful for understanding camera-based assistive technology and Braille interaction.
Research on Gesture Fatigue and the Gorilla-Arm Effect
Useful for understanding physical comfort and fatigue during mid-air interaction.
By the end of the thesis, you will have:
Built an AI + Computer Vision system that recognizes hand gestures.
Created a real-world gesture dataset by collecting data from participants.
Designed and conducted an HCI user study and learned how to evaluate interaction using speed, accuracy, error, and comfort.
Built a working gesture-based Braille prototype that you can demonstrate in your portfolio.
Learned the complete research process — from an idea and data collection to AI development, user testing, analysis, and thesis writing.
Potential for research publication in HCI, assistive technology, or AI-related venues, depending on the quality and findings of the study.
