Analytic of image posts on social media for hate speech

Abstract

Social media platforms have been developing rapidly for the last thirteen years. Billions of people communicate their ideas, information and other expressions with each other through social media. Recently most of the fake news and hate speech have spread by image posts and videos in Facebook, YouTube and Twitter platforms within Sri Lankan community. There is a vital need for obtaining the meaning of the image posts automatically. This research has been supported to the above solution and done to automatically separation and identification of Sinhala and Singlish text in image posts and video thumbnails in social media platforms. This research has three research module components. The first component is the main research component. It is the extraction of Sinhala and English text in social media images. Image posts and video thumbnails were acquired from Facebook and YouTube social media platforms which are on public. Three main algorithms and another three supporting algorithms have been derived to complete this component. They are text detection, text glow removing, text line segmentation, touching character segmentation, Sinhala English character separation and Sinhala English character recognition. The study has achieved the text line segmentation accuracy of 90% for 1,669 text lines and the overall touching character segmentation accuracy of 98% for 58,403 characters and the overall script separation accuracy of 93% for 22,383 script images. A new feature vector was formed in this study and 11,088 Sinhala script images were trained in a sequential model neural network. The accuracy of printed Unicode Sinhala character recognition is higher than the accuracy of printed non-Unicode and handwritten Sinhala character recognition. In the second component, a simple algorithm has been derived to identify the status of social issues in images which is connector module for the main research group work. In the third component, it is tested with three methods to identify key attributes of images to detect redistribution of images within social media and selected the Brute-Force Matcher with ORB descriptors as the highest accuracy obtained by this method.