Explainable artificial intelligence for speech emotion recognition

dc.contributor.advisorThayasivam, U
dc.contributor.authorPatabendige, SSJ
dc.date.accept2025
dc.date.accessioned2026-11-09T04:48:01Z
dc.date.issued2025
dc.description.abstractThe rapid advancement of Artificial Intelligence (AI) has led to its widespread adoption across multiple domains; however, the inherent opacity of conventional AI models has raised concerns regarding their interpretability and trustworthiness. In the context of Speech Emotion Recognition (SER), this lack of transparency limits understanding of how specific acoustic features influence emotional predictions. This research addresses this challenge by developing an explainable and human-centered framework for SER that prioritizes interpretability over classification accuracy. A comprehensive feature extraction pipeline was designed to derive eighteen interpretable audio features from raw speech signals, grouped into five functional categories: Acoustic, Temporal, Spectral, Harmonic/Tonal, and Spectral Energy Distribution. Traditional machine learning models Artificial Neural Network (ANN), Support Vector Machine (SVM), Random Forest, XGBoost, and k-Nearest Neighbors (k-NN) were employed to classify seven emotional states: Calm, Happy, Sad, Angry, Fearful, Surprised, and Disgust. Two model-agnostic interpretability techniques, SHapley Additive explanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME), were used to quantify the contribution of each feature to model predictions. A novel Aggregated Feature Importance (AFI) metric was introduced to combine interpretability outcomes by weighting each model’s contribution according to its predictive performance. The analysis revealed noticeable divergence in feature importance rankings when aggregating across individual models, reflecting their inherent learning differences. However, as additional models were incorporated into the ensemble, these variations gradually diminished, showing a clear convergence and increased stability in AFI-based rankings. Key features such as mfcc_mean, mfcc_std, loudness, zero-crossing rate, and etc. consistently emerged as dominant predictors across emotional classes. Statistical validation using independent t-tests confirmed significant differences (p < 0.05) between emotional states, reinforcing the discriminative reliability of these features. The resulting explainable feature–emotion map provides a transparent interpretation of how specific speech characteristics relate to distinct emotional states, supporting greater trust and usability in emotion-aware AI systems. This work establishes a reproducible framework for feature-level explainability in SER and offers a foundation for future research integrating reverse-engineered and temporally adaptive explainable models.
dc.identifier.accnoTH6279
dc.identifier.citationPatabendige, S.S.J, (2025). Explainable artificial intelligence for speech emotion recognition [Master's theses, University of Moratuwa]. Institutional Repository University of Moratuwa. https://dl.lib.uom.lk/handle/123/25909
dc.identifier.degreeMSc in Data Science and Artificial Intelligence
dc.identifier.departmentDepartment of Computer Science & Engineering
dc.identifier.facultyEngineering
dc.identifier.urihttps://dl.lib.uom.lk/handle/123/25909
dc.language.isoen
dc.subjectEXPLAINABLE ARTIFICIAL INTELLIGENCE
dc.subjectARTIFICIAL INTELLIGENCE-Speech Emotion Recognition
dc.subjectMACHINE LEARNING
dc.subjectMACHINE LEARNING-SHapley Additive Explanations (SHAP)
dc.subjectEXPLAINABLE ARTIFICIAL INTELLIGENCE-Local Interpretable Model-Agnostic Explanations (LIME)
dc.subjectDATA SCIENCE AND ARTIFICIAL INTELLIGENCE-Dissertations
dc.subjectCOMPUTER SCIENCE AND ENGINEERING-Dissertations
dc.subjectMSc in Data Science and Artificial Intelligence
dc.titleExplainable artificial intelligence for speech emotion recognition
dc.typeThesis-Full-text

Files

Original bundle

Now showing 1 - 3 of 3
Loading...
Thumbnail Image
Name:
TH6279-1.pdf
Size:
1.1 MB
Format:
Adobe Portable Document Format
Description:
Pre-text
Loading...
Thumbnail Image
Name:
TH6279-2.pdf
Size:
348.02 KB
Format:
Adobe Portable Document Format
Description:
Post-text
Loading...
Thumbnail Image
Name:
TH6279.pdf
Size:
3.5 MB
Format:
Adobe Portable Document Format
Description:
Full-thesis

License bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
license.txt
Size:
1.71 KB
Format:
Item-specific license agreed upon to submission
Description: