Co-word analysis based automatic web search

Abstract

Automatic searching, knowledge acquisition and question answering are crying needs among the contemporary World Wide Web users. However conventional web is the major barriers for realizing above applications. This is because almost all important information in it is in natural languages and natural languages are very hard to be manipulated and understood by a computer. As a solution, more than a decade ago, semantic web was introduced, there was a lot of hope on machine understandability of the web. However the semantic web is still very far from realization due to the effort required for semantic tagging of available information. In this project we try to build a solution for automatic searching in conventional web and similar information sources by mimicking the human natural language learning and knowledge representation process. Our approach is based on the hypothesis, which inspired by popular philosophy of science, that learning is matching known facts with new facts. In the context of conventional web, we employ statistical natural language processing technique co-word analysis for matching already available facts with new facts collected during automatic searching process. As a proof of above hypothesis we have built a personalized automatic knowledge extraction application. That extracts knowledge from conventional web or similar information source regarding user queries and present synthesized documents related to the knowledge area of the query. Evaluation done by manual comparison of documents produced by the application and a document produced by a human user by web searching. Results showed automatic knowledge acquisition performs acceptable manner in most of the situations.