Abstract:

Lip reading, the process of interpreting speech by visually analyzing lip movements, has gained significant attention in recent years due to its potential applications in various domains, including speech recognition, assistive technologies, and human-computer interaction. However, most existing lip reading systems have primarily focused on English or other widely spoken languages, leaving a considerable gap in the research for less-represented languages like Arabic. In this work, we present a new dataset for Arabic language lip reading that captures a wide range of variability between various speakers. Then, a lip-reading system design is proposed, employing a 3D Convolutional Neural Network (CNN) and Bidirectional-LSTM (Bi-LSTM) architecture. The proposed system addresses the challenges posed by the unique phonetic characteristics and distinct lip movements of the Arabic language. By leveraging the temporal information encoded in the 3D CNN model, the system captures both spatial and temporal dependencies in the lip motion sequence, enabling more accurate lip reading performance. Experimental results are presented to show the effectiveness of the proposed model compared to the state of the art.

Speaker Bio :

Mr. Adeb Ali Magad received his B.Sc. degree from Electrical Engineering Department at KFUPM, and his M.Sc. degree from the Control and Instrumentation Engineering Department at KFUPM, where he is currently pursuing his Ph.D. degree. His research interests are mainly on the applications of nonlinear filtering, computer vision, and SLAM on Autonomous Driving and Robotics. His Area of Specialization includes Control/Autonomous Driving.