Have a personal or library account? Click to login
A Survey on Job Scheduling in Big Data Cover

A Survey on Job Scheduling in Big Data

Cybernetics and Information Technologies
Volume 16 (2016): Issue 3 (September 2016)
By: M. Senthilkumar and  P. Ilango  
Open Access
|Aug 2016

References

  1. 1. Abraham, A., R. Buyya, B. Nath. Nature’s Heuristics for Scheduling Jobs on Computational Grids. - In: Proc. of IEEE International Conference on Advanced Computing and Communications, 2000, pp. 1-8.
    Search in Google Scholar
  2. 2. Ada, R., R. Kaur. - International Journal of Advanced Research in Computer Science and Software Engineering. - Ijarcsse, Vol. 3, 2013, No 3, pp. 665-668.
    Search in Google Scholar
  3. 3. Assunção, M. D., R. N. Calheiros, S. Bianchi, M. A. S. Netto, R. Buyya. Big Data Computing and Clouds: Trends and Future Directions. - Journal of Parallel and Distributed Computing, 2015, 79-80, pp. 3-15.10.1016/j.jpdc.2014.08.003
    Search in Google Scholar
  4. 4. Bardhan, S., D. Menasc. The Anatomy of Mapreduce Jobs, Scheduling, and Performance Challenges. - In: Proc. of Computer Measurement Group, 2013.
    Search in Google Scholar
  5. 5. Bardhan, S., D. Menascé. Queuing Network Models to Predict the Completion Time of the Map Phase of Map Reduce Jobs, 2012.
    Search in Google Scholar
  6. 6. Bu, Y., B. Howe, M. D. Ernst. Ha Loop : Efficient Iterative Data Processing on Large Clusters. - Proceedings of the VLDB Endowment, Vol. 3, 2010, No 1-2, pp. 285-296.10.14778/1920841.1920881
    Search in Google Scholar
  7. 7. Casavant, T. L., J. G. Kuhl. A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems. - IEEE Transactions on Software Engineering, Vol. 14, 1988, No 2, pp. 141-154.10.1109/32.4634
    Search in Google Scholar
  8. 8. Coulouris, G., J. Dollimore, T. Kindberg. Distributed Systems: Concepts and Design. - Computer, Vol. 4, 2012.
    Search in Google Scholar
  9. 9. Dabhi, V. K., H. B. Prajapati. Soft Computing Based Intelligent Grid Architecture. - In: Proc. of International Conference on Computer and Communication Engineering (ICCCE’08), Global Links for Human Development, 13-15 May 2008, pp. 574-577.
    Search in Google Scholar
  10. 10. Davis, R. I., A. Burns. A Survey of Hard Real-Time Scheduling for Multiprocessor Systems. - ACM Computing Surveys, Vol. 43, 2011, No 4, pp. 1-44.10.1145/1978802.1978814
    Search in Google Scholar
  11. 11. Dittrich, J., J.-A. Quiané-Ruiz, A. Jindal, Y. Kargin, V. Sett y, J. Schad. Hadoop++: Makinga Yellow Elephant Run Likea Cheetah (Without it Even Noticing). - Proceedings of the VLDB Endowment, Vol. 3, 2010, No 1-2, pp. 515-529.10.14778/1920841.1920908
    Search in Google Scholar
  12. 12. Dong, F., S. G. Akl. Scheduling Algorithms for Grid Computing : State of the Art and Open Problems. - Components, Vol. 202, 2006, No 4, pp. 1-55.
    Search in Google Scholar
  13. 13. Doulkeridis, C., K. Nørvåg. A Survey of Large-Scale Analytical Query Processing in Map Reduce. - VLDB Journal, Vol. 23, 2014, No 3, pp. 355-380.10.1007/s00778-013-0319-9
    Search in Google Scholar
  14. 14. Ekanayake, J., H. Li, B. Zhang, T. Gunarathne, S. Bae, J. Qiu, G. Fox. Twister : A Runtime for Iterative Map Reduce. - In: Proc. of 19th ACM International Symposium on High Performance Distributed Computing, HPDC’10, 2010, pp. 810-818.
    Search in Google Scholar
  15. 15. Hadoop Fair Scheduler Design Document, 2010, pp. 1-11
    Search in Google Scholar
  16. 16. He, C., Y. Lu, D. Swanson. Matchmaking: A New Map Reduce Scheduling Technique. - In: Proc. of 3rd IEEE International Conference on Cloud Computing Technology and Science, Cloud Com’2011, 2011, pp. 40-47.
    Search in Google Scholar
  17. 17. Herodotou, H., S. Babu. Profiling, What-if Analysis, and Cost-Based Optimization of Map Reduce Programs. - PVLDB: Proceedings of the VLDB Endowment, Vol. 4, 2011, No 11, pp. 1111-1122.10.14778/3402707.3402746
    Search in Google Scholar
  18. 18. Herodotou, H., F. Dong, S. Babu. Mapreduce Programming and Costbased Optimization? Crossing this Chasm with Starfish. - Proceedings of the VLDB Endowment, Vol. 4, 2011, No 12, pp. 1446-1449.10.14778/3402755.3402792
    Search in Google Scholar
  19. 19. Jahani, E., M. J. Cafarella, C. Ré. Automatic Optimization for Map Reduce Programs. - Proceedings of the VLDB Endowment, Vol. 4, 2011, No 6, pp. 385-396.10.14778/1978665.1978670
    Search in Google Scholar
  20. 20. Gautam, J. V., H. B. Prajapati, V. K. Dabhi, S. Chaudhary. A Survey on Job Scheduling Algorithms in Big Data Processing. - In: Proc. of IEEE International Conference on Electrical, Computer and Communication Technologies (ICECCT’15), Coimbatore, 2015, pp. 1-11.10.1109/ICECCT.2015.7226035
    Search in Google Scholar
  21. 21. Kc, K., K. Anyanwu. Scheduling Hadoop Jobs to Meet Deadlines. - In: Proc. of 2nd IEEE International Conference on Cloud Computing Technology and Science (Cloud Com’10), 2010, pp. 388-392.10.1109/CloudCom.2010.97
    Search in Google Scholar
  22. 22. Kwon, Y., M. Balazinska, B. Howe, J. Rolia. Skew Tune: Mitigating Skew in Mapreduce Applications. - In: Proc. of 2012 ACM SIGMOD International Conference on Management of Data, 2012, pp. 25-36.
    Search in Google Scholar
  23. 23. Lee, K. H., Y. J. Lee, H. Choi, Y. D. Chung, B. Moon. Parallel Data Processing with Map Reduce: A Survey. - SIGMOD Record, Vol. 40, 2011, No 4, pp. 11-20.10.1145/2094114.2094118
    Search in Google Scholar
  24. 24. Liao, Xinyi, et al. An Enforcement of Real Time Scheduling in Spark Streaming. - In: Proc. of Green Computing Conference and Sustainable Computing Conference IEEE, 2015, pp. 1-6.
    Search in Google Scholar
  25. 25. Lim, H., H. Herodotou, S. Babu. Stubby: A Transformation-Based Optimizer for Map Reduce Workflows. - Proceedings of the VLDB Endowment, Vol. 5, 2012, No 11, pp. 1196-1207.10.14778/2350229.2350239
    Search in Google Scholar
  26. 26. Lin, Y., D. Agrawa l, C. Chen, B. C. Ooi, S. Wu. Llama: Leveraging Columnar Storage for Scalable Join Processing in the Map Reduce Framework. - In: Proc. of 2011 International Conference on Management of Data (SIGMOD’11), 2011, pp. 961-972.
    Search in Google Scholar
  27. 27. Prajapati, H. B., V. A. Shah. Scheduling in Grid Computing Environment. - In: Proc. of 4th International Conference on Advanced Computing & Communication Technologies, 2014, pp. 315-324.10.1109/ACCT.2014.32
    Search in Google Scholar
  28. 28. Rao, B. T., L. S. S. Reddy. Survey on Improved Scheduling in Hadoop Map Reduce in Cloud Environments. - International Journal of Computer Applications, Vol. 34, 2012, No 9, pp. 29-33.
    Search in Google Scholar
  29. 29. Rasooli, A., D. G. Down. A Hybrid Scheduling Approach for Scalable Heterogeneous Hadoop Systems. - In: Proc. of 2012 SC Companion: High Performance Computing, Networking Storage and Analysis (SCC’2012), 2012, pp. 1284-1291.
    Search in Google Scholar
  30. 30. Sakr, S., A. Liu, A. G. Fayoumi. The Family of Map Reduce and Large-Scale Data Processing Systems. - ACM Computing Surveys, Vol. 46, 2013, No 1, pp. 1-44.10.1145/2522968.2522979
    Search in Google Scholar
  31. 31. International Journal of Advanced Research, Vol. 3, 2013, No 5, pp. 875-878.
    Search in Google Scholar
  32. 32. Suthaharan, S. Big Data Classification: Problems and Challenges in Network Intrusion Prediction with Machine Learning. - Performance Evaluation Review, Vol. 41, 2014, No 4, pp. 70-73.10.1145/2627534.2627557
    Search in Google Scholar
  33. 33. Verma, A., L. Cherkasova, R. Campbell. ARIA: Automatic Resource Inference and Allocation for Map Reduce Environments. - In: Proc. of 8th ACM International Conference on Autonomic Computing, 2011, pp. 235-244.
    Search in Google Scholar
  34. 34. Wang, C. Journal of Computers, Vol. 8, 2013, No 3.
    Search in Google Scholar
  35. 35. Wolf, J., A. Balmin, D. Rajan, K. Hildrum, R. Khandekar, S. Parekh, R. Vernica. CIRCUMFLEX: A Scheduling Optimizer for Map Reduce Workloads with Shared Scans. - ACM SIGOPS Operating Systems Review, Vol. 46, 2012, No 1.10.1145/2146382.2146388
    Search in Google Scholar
  36. 36. Yang, Zhiw Ei, et al. Adaptive Task Scheduling Strategy for Heterogeneous Spark Cluster. - Computer Engineering, 2016.
    Search in Google Scholar
  37. 37. Yong, M., N. Garegrat, S. Mohan. Towardsa Resource Aware Scheduler in hadoop. - In: Proc. of ICWS, 2009, pp. 1-10.
    Search in Google Scholar
  38. 38. Yoo, D., K. M. Sim. A Comparative Review of Job Scheduling for Map Reduce. - In: IEEE International Conference on Cloud Computing and Intelligence Systems, 2011, pp. 353-358.10.1109/CCIS.2011.6045089
    Search in Google Scholar
  39. 39. Zaharia, M., D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, I. Stoica. Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling. - In: Proc. of 5th European Conference on Computer Systems, 2010, pp. 265-278.
    Search in Google Scholar
  40. 40. Zaharia, M., A. Konwinski, A. Joseph, R. Katz, I. Stoica. Improving Map Reduce Performance in Heterogeneous Environments. - Osdi, 2008, pp. 29-42.
    Search in Google Scholar
  41. 41. Zhang, Y., Q. Gao, L. Gao, C. Wang. i Map Reduce: A Distributed Computing Framework for Iterative Computation. - Journal of Grid Computing, Vol. 10, 2012, No 1, pp. 47-68. 10.1007/s10723-012-9204-9
    Search in Google Scholar
DOI: https://doi.org/10.1515/cait-2016-0033 | Journal eISSN: 1314-4081 | Journal ISSN: 1311-9702
Journal RSS Feed
Language: English
Page range: 35 - 51
Published on: Aug 19, 2016
Published by: Bulgarian Academy of Sciences, Institute of Information and Communication Technologies
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year
Keywords:
Big Data,
Hadoop,
MapReduce,
classification,
HDFS (Hadoop Distributed File System),
scheduler
Related subjects:
Computer sciences,
Information technology

© 2016 M. Senthilkumar, P. Ilango, published by Bulgarian Academy of Sciences, Institute of Information and Communication Technologies
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Previous articleVolume 16 (2016): Issue 3 (September 2016)Next article