Computer, read my lips

A computer that can lip-read would be a boon for those with speech difficulties as well as people in noisy environments where conventional voice recognition cannot be used. Now, a team at the University of Essex, in Colchester, UK, and colleagues at the Chongqing University of Posts and Telecommunications, in Chongqing, China, have developed an accurate system that can analyse and interpret lip shape.

Writing in the International Journal of Modelling, Identification and Control, University of Essex professor of computer science and electronic engineering, Huosheng Hu, and colleagues explain why conventional voice-activated human-machine interfaces are not available to many people, either because of personal disability or the environment in which such systems must be used.

The team explains that their system is based on a novel lip shape feature extraction method that uses the computer’s web camera to assimilate an image sequence of the user’s lips. A statistical method known as a hybrid dual-tree complex wavelet transform (DT-CWT) and discrete cosine transform (DCT) are then used to pin down the characteristics of the lip shapes at different points in time, which is then coupled to a trained database of shapes associated with specific words or commands.

The approach sidesteps the problem of head movement by pinning down invariant points on the images around the mouth. The team has tested the system with a wheelchair user and demonstrated that a small vocabulary of commands to control the wheelchair can be recognised in a noisy environment once the system is trained.

A computer system that could lip read might also be adapted for mobile devices such as smart phones adding an extra layer to applications that respond to voice commands in environments where the user must remain quiet if not silent, such as libraries, theatres and even monasteries.

“Hybrid lip shape feature extraction and recognition for human-machine interaction” in Int. J. Modelling, Identification and Control, 2013, 18, 191-198

Computer, read my lips is a post from: David Bradley's Science Spot

via Science Spot http://sciencespot.co.uk/computer-read-my-lips-2.html