DISTRIBUTED QUERY PROCESSING ON THE DEEP WEB

Abstract

The WWW is a ubiquitous, open, large heterogeneous, distributed hypertext system. It has become major means of disseminating information. The openness and uncontrolled growth of the web has posed problems in searching or querying the web for relevant information. Currently searching information on the web involves use of search engines which use keywords supplied by the users to search through a large index of static web pages. Most of the webs information is hidden on dynamically generated sites and standard search engines do not find it as they cannot index dynamically generated web pages. In this dissertation, the design requirements for searching the relevant web documents in dynamically generated web pages are identified. A scheme based on Distributed query processing is then proposed which finds the relevant documents. Then an optimization of a word relevance algorithm is proposed which orders the documents based on their relevance to the query. Simulations for the proposed scheme are conducted for the different user queries. Finally the thesis is concluded by pointing out some open issues and possible direction of future research relating to query processing technique and relevance algorithm