A fine-grained scheduling strategy for improving the performance of parallel frequent itemsets mining

Chao Chin Wu, Lien Fu Lai, Liang Tsung Huang, Syun Sheng Jhan, Chung Lu

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

We propose a scheduling strategy in this paper to address the load imbalance problem of the distributed parallel apriori (DPA) algorithm published recently. We use fine grained tasks that are derived by dividing the tasks defined by DPA into smaller subtasks. The subtasks will be scheduled by a dynamic self-scheduling scheme for better load balance. Furthermore, we propose two different methods for data transmission from the master to workers. The first one broadcasts all the frequent k-itemsets to all work nodes while the second one transmits only the required data to each individual work node. Experimental results demonstrate the proposed two approaches both outperform DPA. The first one is more suitable for small datasets and the second one provides steadier performance improvement no matter which self-scheduling scheme is adopted.

Original languageEnglish
Pages (from-to)264-274
Number of pages11
JournalInternational Journal of Computational Science and Engineering
Volume6
Issue number4
DOIs
Publication statusPublished - 2011 Dec 1

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Software
  • Modelling and Simulation
  • Hardware and Architecture
  • Computational Mathematics
  • Computational Theory and Mathematics

Cite this