Table of Contents Author Guidelines Submit a Manuscript
Mathematical Problems in Engineering
Volume 2015 (2015), Article ID 408921, 7 pages
http://dx.doi.org/10.1155/2015/408921
Research Article

A Replication-Based Mechanism for Fault Tolerance in MapReduce Framework

College of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China

Received 20 October 2014; Accepted 31 January 2015

Academic Editor: Hui-Huang Hsu

Copyright © 2015 Yang Liu and Wei Wei. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

MapReduce is a programming model and an associated implementation for processing and generating large data sets with a parallel, distributed algorithm on a cluster. In cloud environment, node and task failure are no longer accidental but a common feature of large-scale systems. Current rescheduling-based fault tolerance method in MapReduce framework failed to fully consider the location of distributed data and the computation and storage overhead of rescheduling failure tasks. Thus, a single node failure will increase the completion time dramatically. In this paper, a replication-based mechanism is proposed, which takes both task and node failure into consideration. Experimental results show that, compared with default mechanism in Hadoop, our mechanism can significantly improve the performance at failure time, with more than 30% decreasing in execution time.