IIT Jodhpur & AIIMS Jodhpur’s Researchers Create Communication Device for People with Speech Impairment

People with speech disabilities, even those that know sign language, face a tough time while going about their daily routine. Sign language isn’t easily understood by everyone they encounter in their day-to-day life and this often makes them dependent on others for communicating. Newly-developed applications and gadgets have tried to solve their travails, but a majority of these technologies are in the nascent stage and the apps reliant on mobile network offer limited functionalities due to their dependence on the Internet. However, in what could prove a major boon for people with speech impediments, innovators from the Indian Institute of Technology (IIT) Jodhpur and All India Institute Of Medical Science (AIIMS) Jodhpur have jointly developed low-cost ‘Talking Gloves’.

These talking gloves use principles of Artificial Intelligence (AI) and Machine Learning (ML) to automatically generate speech that is language independent and helps speech-impaired people communicate more effectively. The developed device can help individuals convert hand gestures into text or pre-recorded voices, thereby helping them convey their message effectively and giving them independence. And, most importantly, the developed device costs less than Rs 5,000 making it affordable for a large section of its user-base.

How the ‘Talking Gloves’ work:

In the developed device electrical signals are generated by a first set of sensors, wearable on a combination of a thumb, finger(s), and/or a wrist of a first hand of a user. These electrical signals are produced by the combination of fingers, thumb, hand and wrist movements. Similarly, electrical signals are also generated by the second set of sensors on the other hand.

These electrical signals are received at a signal processing unit. The magnitude of the received electrical signals is compared with a plurality of pre-defined combinations of magnitudes stored in a memory by using the signal processing unit. By using Artificial Intelligence and Machine Learning algorithms, these combinations of signals are translated into phonetics corresponding to at least one of consonant and a vowel. For instance, the consonant and the vowel can be from Hindi language phonetics. A phonetic is assigned to the received electrical signals based on the comparison.

An audio signal is generated by an audio transmitter corresponding to the assigned phonetic and based on trained data associated with vocal characteristics stored in a machine learning unit. The generation of audio signals according to the phonetics having combination of vowels and consonants leads to the generation of speech and enables the mute people to audibly communicate with others. The speech synthesis technique of the present subject matter uses phonetics, and therefore the speech generation is independent of any language.

Expressing his views on the innovation, Professor Sumit Kalra, Assistant Professor, Department of Computer Science and Engineering – IIT Jodhpur, said, “The language-independent speech generation device will bring the people back to the mainstream in today’s global era without any language barrier. Users of the device only need to learn once and they would be able to verbally communicate in any language with their knowledge. Additionally, the device can be customized to produce a voice similar to the original voice of the patients which makes it appear more natural while using the device.”

Various preordained situations, diseases or injuries deprive people of their natural ability to communicate verbally. In recent years, the technological advancement in the field of electro-medical devices for life support, implantable biomedical devices, and wearable medical devices have successfully provided artificial abilities to afflicted people related to any kind of disability or impairment.

The patented innovation from IIT Jodhpur and AIIMS Jodhpur is a big step towards advancement in this field. The team of scientists and innovators is further working to enhance the features such as durability, weight, responsiveness, and ease-of-use of the developed device. The developed product will be commercialised through a startup incubated by IIT Jodhpur. A patent for this innovation has been acquired by Professor Sumit Kalra (Assistant Professor, Department of Computer Science and Engineering – IIT Jodhpur), along with his team of innovators including Dr. Arpit Khandelwal from IIT Jodhpur and Dr. Nithin Prakash Nair (SR, ENT), Dr. Amit Goyal (Professor & Head, ENT) and Dr. Abhinav Dixit (Prof., Dept. of Physiology) from AIIMS Jodhpur.

Source: Financial Express