Semantic event detection via multimodal data mining

Abstract

Anovel framework is presented for video event detection. The core of the framework is an advanced temporal analysis and multimodal data mining method that consists of three major components: low-level feature extraction, temporal pattern analysis, and multimodal data mining. A set of visual/audio features is first extracted with the aid of little domain knowledge. Next, the temporal pattern analysis step is conducted to systematically search for the optimal temporal patterns that are significant for characterizing the events and to perform a data reduction operation to boost the data mining perfocomponent. One of the unique characteristics of this framework is that it offers strong generality and extensibility with the capability of exploring representative event patterns with little human interference. The framework is presented with its application to the detection of the soccer goal events over a large collection of soccer video data with various production styles.rmance. Finally, the events of interest are detected automatically in the data mining