Product attribute extraction based real-time c2c matching of microblogging messages.

Abstract

We describe a solution for real-time matching of microblogging messages related to product selling or buying. C2C buy/sell interest matching in real time is nontrivial due to the complexities of interpreting social media messages, number of messages, and diversity of products/services. Therefore, we adopt a combination of techniques from natural language processing, complex event processing, and distributed systems. First, we convert the message into semantics using named-entity recognition with CRF and Logistic Regression. Then the extracted data are matched using a complex event processor. Moreover, NoSQL and inmemory computing are used to enhance the scalability and performance. The proposed solution shows a high accuracy where classification and CRF models recorded an accuracy of 98.5% and 82.07% when applied to a real-world dataset. Low latency was observed for information extraction, in-memory data manipulation, and complex event processing were latencies were 0.5 ms, 5 ms, and 3.6 ms, respectively.