Gesture Gloves

Product design | Hardware Prototyping | Arduino Programming

Problem Statement

Speechless patients often find it difficult to convey what they require at a particular moment. This makes it impossible for the person assisting them to understand what help is needed by the patient. In such circumstances, if the objects of reference are out of sight, he or she cannot point out what is required. Difficulties arise when the need is abstract and cannot be conveyed just by pointing at it. Thus there is need for a device that can facilitate easy, clear and distant communication for the speechless users.

Project Details

Project Group:

Yashvi Upadhyaya(me)
Maitri Thaker
Gargi Vyas
Neha Udeshi

This document is a representation of my individual contributions to the project. This includes:

Researching about the problem
Collection and analysis of Images of the fingers at rest
Hardware Prototyping(Glove) and Arduino programming(code on Github)
Designing the Gesture vs Command chart

The project and research paper titled "Gesture Gloves" was presented by our group, in front of the National Assessment and Accreditation Council(NAAC) to represent our college. The following document describes the process that I followed to design a prototype for the glove.

User Research

As mentioned in the project brief, we had some idea about the problem. However, to know more about the problem we choose to qualitatively collect data through field observations by visiting hospitals and meeting various speechless patients. Following were the major observations made:

Speechless patients need a sound which includes banging on a surface, striking sound-producing objects like a metal spoon, a bell, etc to call for help.

Sometimes it becomes difficult to convey what exactly one needs especially when it is not possible to simply point at whatever is needed.

The communication becomes difficult when the patient cannot move his/her hands for pointing out at the objects needed or use a bell, etc.

Brainstorming

With the observations made we started searching for ideas by jotting down all possible solutions. Out of all, we came up with 2 ideas.

Buttons

Gesture Glove

We made a paper prototype for the button board and simulated audio for every button. For the gesture glove, we asked the user to wear a glove and refer to a table to make gestures. Each gesture gave an audio output.

Button Board	Gesture Glove
Only possible when there are less instructions. Otherwise, it becomes cluttered and confusing.	Gives 2^n solutions for n fingers. Only one needs is to refer to a chart of gesture vs command.
The position is static, need to move towards the switchboard.	Position not static. Feels like a part of the body, only fingers are needed to be folded for giving commands.
Direct commands - Low cognitive load on the user.	Need to know the corresponding command. Initially, refer to chart then one can remember those. High cognitive load on the user at the beginning.

Design Challenge

A major drawback that exists in all current systems is that the user needs to be conscious about their finger bends. There can be unnecessary outputs in such systems as the patient might unconsciously bend his/her fingers while resting. Therefore it becomes necessary to make the system, considering the natural bends of the fingers and the bends when the patient is resting.

Solution

Pictures of fingers of people resting and sleeping were taken. From this, it was concluded that naturally, the fingers are never straight or completely curled. Every finger has 3 joints which produce 3 different angles when the finger is bent. These three angles are at the DIP, PIP and MCP joints- as shown in the figure. Out of this three joints, the angle at DIP cannot be identified distinctly as it varies slowly with the bend of fingers. The angle at MCP is also not a good option to be considered as the input trigger, because it may or may not be bended along with the fingers by the user of the glove.

Here, the angle at PIP is the apt choice as it proportionally varies with the bend of the finger. A dataset of fingers at resting positions was collected and the angle of at the PIP joint was calculated. All these bend angles fell in a range of 30 degrees to 50 degrees. Thus, for our system, a threshold angle of 60 degrees was chosen, beyond which all the user commands will be considered and the output will be given.

System design

The system architecture was designed as follows:

Prototyping the glove

Flex sensors being lightweight and flexible were used in the glove to measure the angle when the finger is bent. Flex sensors are available in different lengths and therefore choosing the correct length is important because if the sensor is long then it will cover the entire finger which would lead to errors in angle calculated as the DIP angle will also be considered. To avoid this, the human finger was studied. I sketched fingers of variable lengths to measure the distance from the end of MCP joint till the start of DIP joints. This area of the finger when covered by the flex sensor would only calculate the PIP angle and give the intended result.

Note: The below diagram is just to explain how the length was calculated. While performing the study, the fingers of the people of different age and gender were considered.

Flex sensors are available in 2 lengths 2.2inches and 4.5inches. From the above observations, we could conclude that a flex sensor of the length of 2.2 inches would be the most effective length to give accurate results. Other components used to prototype the glove are:

Arduino Uno (ATmega328P)
Bluetooth Module HC-05
Connecting wires

Final Prototype

Having designed the system architecture and the connections, we moved ahead to develop a prototype of this system. The prototype was in working condition and was tested on multiple users. It was observed that the patients could easily convey their needs using the gloves and the gesture vs command chart. This improved the overall effectiveness in communication. Also, after a point the patient started remembering the gestures, removing their dependency on the reference chart.

View Code

What did learn?

This was my first time working on a hardware prototyping project. I realised that to design wearables it is very important to study the physical aspects of the users. This is a major challenge to be tackled as no human body is the same and therefore the design of the product should be robust to be usable by the users irrespective of the variations in the physical characteristics of the human body. (in this case fingers)

I understood that user research forms a major part while rolling out solutions. Understanding what the user needs and what are their constraints helped us define a solution.

I understood the importance of brainstorming to come up with innovative solutions. Jotting down multiple possibilities for selecting the most efficient and feasible solution is the phase where a lot of research was required which added to my knowledge.

This is a design thinking and prototyping casestudy. From the observations made while testing the glove, it was concluded that the glove satisfies the user needs and eases communication. This system can be enhanced with technologies like artificial intelligence which would enable one to personalise the experience of using the glove.