FREQUENT ITEMSET MINING FRAMEWORK USING MPI-STYLE, WSRF SERVICES BASED ON WEKA4WS

Abstract

In many fields of research and applications large amounts of data are being generated, collected and stored incrementally. In such scenarios Parallel and Distributed Data Mining (PDDM) is becoming a more and more crucial area for research and applications. The concept of data mining in Grid environment allows the data mining process to be deployed and used in a grid environment where data and service resources are geographically distributed across multiple and geographically distributed administrative domains. The Web Services Resource Framework (WSRF) is the standard for the implementation of Grid applications which can be exploited for developing high-level services for distributed data mining applications. More performance can be achieved if there is support for tightly-coupled services where running services can exchange messages with each other as per MPI standards. This dissertation report presents the design and development of an efficient frequent itemset mining framework for mining incremental and distributed data on Grid, integrated with MPI programming technologies of MPI-Style Web Services (MPIWS). MPIWS takes advantage of the SOAP communication protocol, and allows direct MPI-Style communication among loosely coupled services. The proposed framework generates local models as well as global model of incremental frequent itemset mining. Both of these models are stored in WSRF stateful resource and used in subsequent mining over incremented dataset. The most important feature of proposed framework is that it is fully compliant with WSRF specifications. It has been evaluated for its performance analysis with various Grid configurations and dataset increment sizes. The obtained results validate the feasibility and efficiency of MPI style WSRF services in Grid environment for tightly-coupled data mining applications. Keywords : Data Mining, Frequent Itemset Mining, Grid Computing, Globus Toolkit, WSRF, MPIWS, ZIGZAG. iv