AI-Based Early Detection of Neurological and Mental Disorders using Multimodal Deep Learning

Abstract

Early diagnosis of neurological and mental health disorders is a pressing issue in the current healthcare system, especially in underdeveloped areas with limited access to advanced diagnostic tools and healthcare professionals. Diseases like Parkinson’s, depression and stroke often present with early signs that are difficult to spot through conventional medicine. In this research, we propose the use of a new multimodal deep learning model with which non-invasive as well as economically affordable prediction of disease can be done timely with the use of speech along with facial micro-expressions. Our work exploits Convolutional Neural Networks (CNN) for spatial feature extraction from facial images and Recurrent Neural Networks (RNN/LSTM) for temporal speech feature modelling. A fusion strategy is employed to combine multi-modal features for enhanced model accuracy and reliability. The goal of this strategy is to identify people at different risk levels to catch it early and keep an eye on their health for a long time to come. In addition, the system's ability to lessen the need for expensive diagnosis tests proves it can be used in the real world, especially in rural areas. We also discuss the issues of privacy, dataset collection, and generalisation. The method has the potential to revolutionize preventive health by facilitating scalable, affordable, and smart health diagnostic and monitoring systems.