On the optimal number of computational resources in MapReduce

Htway Htway Hlaing; Hidehiro Kanemitsu; Tatsuo Nakajima; Hidenori Nakazato

doi:10.1007/978-3-030-23502-4_17

On the optimal number of computational resources in MapReduce

Htway Htway Hlaing^*, Hidehiro Kanemitsu, Tatsuo Nakajima, Hidenori Nakazato

^*Corresponding author for this work

School of Fundamental Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Big data computing in the cloud needs faster processing and better resource provisioning. MapReduce is the framework for computing large scale datasets in cloud environments. Optimization of resource requirement for each job to satisfy a specific objective in MapReduce is an open problem. Many factors, e.g., system side information and requirements of each client must be considered to estimate the appropriate amount of resources. This paper presents a mathematical model for the optimal number of map tasks in MapReduce resource provisioning. This model is to estimate the optimal number of the mappers based on the resource specification and the size of the dataset.

Original language	English
Title of host publication	Cloud Computing – CLOUD 2019 - 12th International Conference, Held as Part of the Services Conference Federation, SCF 2019, Proceedings
Editors	Qingyang Wang, Liang-Jie Zhang, Dilma Da Silva
Publisher	Springer-Verlag
Pages	240-252
Number of pages	13
ISBN (Print)	9783030235017
DOIs	https://doi.org/10.1007/978-3-030-23502-4_17
Publication status	Published - 2019 Jan 1
Event	12th International Conference on Cloud Computing, CLOUD 2019 held as part of the Services Conference Federation, SCF 2019 - San Diego, United States Duration: 2019 Jun 25 → 2019 Jun 30

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11513 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	12th International Conference on Cloud Computing, CLOUD 2019 held as part of the Services Conference Federation, SCF 2019
Country/Territory	United States
City	San Diego
Period	19/6/25 → 19/6/30

Keywords

Big data
Cloud computing
Resource provisioning

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-23502-4_17

Cite this

Hlaing, H. H., Kanemitsu, H., Nakajima, T., & Nakazato, H. (2019). On the optimal number of computational resources in MapReduce. In Q. Wang, L-J. Zhang, & D. Da Silva (Eds.), Cloud Computing – CLOUD 2019 - 12th International Conference, Held as Part of the Services Conference Federation, SCF 2019, Proceedings (pp. 240-252). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11513 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-23502-4_17

On the optimal number of computational resources in MapReduce. / Hlaing, Htway Htway; Kanemitsu, Hidehiro; Nakajima, Tatsuo et al.
Cloud Computing – CLOUD 2019 - 12th International Conference, Held as Part of the Services Conference Federation, SCF 2019, Proceedings. ed. / Qingyang Wang; Liang-Jie Zhang; Dilma Da Silva. Springer-Verlag, 2019. p. 240-252 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11513 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Hlaing, HH, Kanemitsu, H, Nakajima, T & Nakazato, H 2019, On the optimal number of computational resources in MapReduce. in Q Wang, L-J Zhang & D Da Silva (eds), Cloud Computing – CLOUD 2019 - 12th International Conference, Held as Part of the Services Conference Federation, SCF 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11513 LNCS, Springer-Verlag, pp. 240-252, 12th International Conference on Cloud Computing, CLOUD 2019 held as part of the Services Conference Federation, SCF 2019, San Diego, United States, 19/6/25. https://doi.org/10.1007/978-3-030-23502-4_17

Hlaing HH, Kanemitsu H, Nakajima T , Nakazato H. On the optimal number of computational resources in MapReduce. In Wang Q, Zhang L-J, Da Silva D, editors, Cloud Computing – CLOUD 2019 - 12th International Conference, Held as Part of the Services Conference Federation, SCF 2019, Proceedings. Springer-Verlag. 2019. p. 240-252. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-23502-4_17

Hlaing, Htway Htway ; Kanemitsu, Hidehiro ; Nakajima, Tatsuo et al. / On the optimal number of computational resources in MapReduce. Cloud Computing – CLOUD 2019 - 12th International Conference, Held as Part of the Services Conference Federation, SCF 2019, Proceedings. editor / Qingyang Wang ; Liang-Jie Zhang ; Dilma Da Silva. Springer-Verlag, 2019. pp. 240-252 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{159e87892456436a9e5b563553c2409e,

title = "On the optimal number of computational resources in MapReduce",

abstract = "Big data computing in the cloud needs faster processing and better resource provisioning. MapReduce is the framework for computing large scale datasets in cloud environments. Optimization of resource requirement for each job to satisfy a specific objective in MapReduce is an open problem. Many factors, e.g., system side information and requirements of each client must be considered to estimate the appropriate amount of resources. This paper presents a mathematical model for the optimal number of map tasks in MapReduce resource provisioning. This model is to estimate the optimal number of the mappers based on the resource specification and the size of the dataset.",

keywords = "Big data, Cloud computing, Resource provisioning",

author = "Hlaing, {Htway Htway} and Hidehiro Kanemitsu and Tatsuo Nakajima and Hidenori Nakazato",

year = "2019",

month = jan,

day = "1",

doi = "10.1007/978-3-030-23502-4_17",

language = "English",

isbn = "9783030235017",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer-Verlag",

pages = "240--252",

editor = "Qingyang Wang and Liang-Jie Zhang and {Da Silva}, Dilma",

booktitle = "Cloud Computing – CLOUD 2019 - 12th International Conference, Held as Part of the Services Conference Federation, SCF 2019, Proceedings",

note = "12th International Conference on Cloud Computing, CLOUD 2019 held as part of the Services Conference Federation, SCF 2019 ; Conference date: 25-06-2019 Through 30-06-2019",

}

TY - GEN

T1 - On the optimal number of computational resources in MapReduce

AU - Hlaing, Htway Htway

AU - Kanemitsu, Hidehiro

AU - Nakajima, Tatsuo

AU - Nakazato, Hidenori

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Big data computing in the cloud needs faster processing and better resource provisioning. MapReduce is the framework for computing large scale datasets in cloud environments. Optimization of resource requirement for each job to satisfy a specific objective in MapReduce is an open problem. Many factors, e.g., system side information and requirements of each client must be considered to estimate the appropriate amount of resources. This paper presents a mathematical model for the optimal number of map tasks in MapReduce resource provisioning. This model is to estimate the optimal number of the mappers based on the resource specification and the size of the dataset.

AB - Big data computing in the cloud needs faster processing and better resource provisioning. MapReduce is the framework for computing large scale datasets in cloud environments. Optimization of resource requirement for each job to satisfy a specific objective in MapReduce is an open problem. Many factors, e.g., system side information and requirements of each client must be considered to estimate the appropriate amount of resources. This paper presents a mathematical model for the optimal number of map tasks in MapReduce resource provisioning. This model is to estimate the optimal number of the mappers based on the resource specification and the size of the dataset.

KW - Big data

KW - Cloud computing

KW - Resource provisioning

UR - http://www.scopus.com/inward/record.url?scp=85068222466&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068222466&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-23502-4_17

DO - 10.1007/978-3-030-23502-4_17

M3 - Conference contribution

AN - SCOPUS:85068222466

SN - 9783030235017

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 240

EP - 252

BT - Cloud Computing – CLOUD 2019 - 12th International Conference, Held as Part of the Services Conference Federation, SCF 2019, Proceedings

A2 - Wang, Qingyang

A2 - Zhang, Liang-Jie

A2 - Da Silva, Dilma

PB - Springer-Verlag

T2 - 12th International Conference on Cloud Computing, CLOUD 2019 held as part of the Services Conference Federation, SCF 2019

Y2 - 25 June 2019 through 30 June 2019

ER -

On the optimal number of computational resources in MapReduce

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this