Have a personal or library account? Click to login
Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce Cover

Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce

Open Access
|Apr 2017

Abstract

Several Job scheduling algorithms have been developed for Hadoop-Map Reduce model, which vary widely in design and behavior for handling different issues such as locality of data, user share fairness, and resource awareness. This article focuses on empirically evaluating the performance of three schedulers: First In First Out (FIFO), Fair scheduler, and Capacity scheduler. To carry out the experimental evaluation, we implement our own Hadoop cluster testbed, consisting of four machines, in which one of the machines works as the master node and all four machines work as slave nodes. The experiments include variation in data sizes, use of two different data processing applications, and variation in the number of nodes used in processing. The article analyzes the performance of the job scheduling algorithms based on various relevant performance measures. The results of the experiments are evident of the performance being affected by the job scheduling parameters, the type of applications, the number of nodes in the cluster, and size of the input data.

DOI: https://doi.org/10.1515/cait-2017-0012 | Journal eISSN: 1314-4081 | Journal ISSN: 1311-9702
Language: English
Page range: 146 - 163
Published on: Apr 6, 2017
Published by: Bulgarian Academy of Sciences, Institute of Information and Communication Technologies
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year

© 2017 Jyoti V. Gautam, Harshadkumar B. Prajapati, Vipul K. Dabhi, Sanjay Chaudhary, published by Bulgarian Academy of Sciences, Institute of Information and Communication Technologies
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.