Development of digital storytelling platform for children based on emotions

Abstract

Storytelling is one of the means to make Gen Alpha children attractive to books. This is becoming a challenging task for parents, as they have to spend time telling stories to children. This research proposed a system that can emotionally tell stories as a human storyteller tells stories with twists and suspense. This research involves the implementation of technologies related to Natural Language Processing and Deep Learning, especially the inclusion of Convolutional Neural Networks, Transformers. Camera images of the storybook are the input for optical character recognition (OCR) of the Storybook. Here, various complex algorithms are used throughout the processes of optical character recognition (OCR), sentence-level contextual emotion recognition(CER), and emotional text to speech(TTS) synthesis to implement the Emotional digital storyteller. In addition, immense testing is carried out to explore the effectiveness and characteristics of the implemented model. We achieve 75% and 72.09% accuracy in OCR and CER respectively and MOS values of 3.85 and 1.65 in neutral and emotional TTS respectively. The experiment results show that each model performs well in the children's domain, specifically storytelling books.