Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment

Hyungjae Jung; Hidenori Nakazato

doi:10.1109/ICDCSW.2014.23

Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment

Hyungjae Jung, Hidenori Nakazato

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Citations (Scopus)

Abstract

MapReduce framework allows users to quickly develop big-data applications and process big-data effectively. However, unexpected malfunction may be found in cloud environment because a distributed system consists of several hardware, and this malfunction often causes delay of overall processing. MapReduce framework provides Speculative Execution (SE). SE reduces delay in a homogeneous environment by assigning delayed tasks to additional nodes. As cloud computing prevails, cloud computing environment is moving from homogeneous to heterogeneous. Original SE is not perfect and sometimes produces inefficient result in a heterogeneous environment. This paper proposes Dynamic Scheduling for Speculative Execution (DSSE) which enhances performance in a heterogeneous environment by improving existing SE. DSSE prevents wasted SE since it calculates processing capability of each node more objectively and precisely. DSSE has reduced entire processing time approximately 10% compared to original SE. Success rate of SE was 100%.

Original language	English
Title of host publication	Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	119-124
Number of pages	6
Volume	30-June-2014
ISBN (Electronic)	9781479941810
DOIs	https://doi.org/10.1109/ICDCSW.2014.23
Publication status	Published - 2014 Aug 29
Event	2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014 - Madrid, Spain Duration: 2014 Jun 30 → 2014 Jul 3

Other

Other	2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014
Country/Territory	Spain
City	Madrid
Period	14/6/30 → 14/7/3

Keywords

Cloud Computing
DSSE
Heterogeneous environment
MapReduce
Speculative Execution

ASJC Scopus subject areas

Computer Networks and Communications
Hardware and Architecture
Software

Access to Document

10.1109/ICDCSW.2014.23

Cite this

Jung, H., & Nakazato, H. (2014). Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment. In Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014 (Vol. 30-June-2014, pp. 119-124). Article 6888850 Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDCSW.2014.23

Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment. / Jung, Hyungjae; Nakazato, Hidenori.
Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014. Vol. 30-June-2014 Institute of Electrical and Electronics Engineers Inc., 2014. p. 119-124 6888850.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Jung, H & Nakazato, H 2014, Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment. in Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014. vol. 30-June-2014, 6888850, Institute of Electrical and Electronics Engineers Inc., pp. 119-124, 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014, Madrid, Spain, 14/6/30. https://doi.org/10.1109/ICDCSW.2014.23

@inproceedings{10d8f44f97a94d5fb2c56b9ea1185615,

title = "Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment",

abstract = "MapReduce framework allows users to quickly develop big-data applications and process big-data effectively. However, unexpected malfunction may be found in cloud environment because a distributed system consists of several hardware, and this malfunction often causes delay of overall processing. MapReduce framework provides Speculative Execution (SE). SE reduces delay in a homogeneous environment by assigning delayed tasks to additional nodes. As cloud computing prevails, cloud computing environment is moving from homogeneous to heterogeneous. Original SE is not perfect and sometimes produces inefficient result in a heterogeneous environment. This paper proposes Dynamic Scheduling for Speculative Execution (DSSE) which enhances performance in a heterogeneous environment by improving existing SE. DSSE prevents wasted SE since it calculates processing capability of each node more objectively and precisely. DSSE has reduced entire processing time approximately 10% compared to original SE. Success rate of SE was 100%.",

keywords = "Cloud Computing, DSSE, Heterogeneous environment, MapReduce, Speculative Execution",

author = "Hyungjae Jung and Hidenori Nakazato",

year = "2014",

month = aug,

day = "29",

doi = "10.1109/ICDCSW.2014.23",

language = "English",

volume = "30-June-2014",

pages = "119--124",

booktitle = "Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014 ; Conference date: 30-06-2014 Through 03-07-2014",

}

TY - GEN

T1 - Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment

AU - Jung, Hyungjae

AU - Nakazato, Hidenori

PY - 2014/8/29

Y1 - 2014/8/29

N2 - MapReduce framework allows users to quickly develop big-data applications and process big-data effectively. However, unexpected malfunction may be found in cloud environment because a distributed system consists of several hardware, and this malfunction often causes delay of overall processing. MapReduce framework provides Speculative Execution (SE). SE reduces delay in a homogeneous environment by assigning delayed tasks to additional nodes. As cloud computing prevails, cloud computing environment is moving from homogeneous to heterogeneous. Original SE is not perfect and sometimes produces inefficient result in a heterogeneous environment. This paper proposes Dynamic Scheduling for Speculative Execution (DSSE) which enhances performance in a heterogeneous environment by improving existing SE. DSSE prevents wasted SE since it calculates processing capability of each node more objectively and precisely. DSSE has reduced entire processing time approximately 10% compared to original SE. Success rate of SE was 100%.

AB - MapReduce framework allows users to quickly develop big-data applications and process big-data effectively. However, unexpected malfunction may be found in cloud environment because a distributed system consists of several hardware, and this malfunction often causes delay of overall processing. MapReduce framework provides Speculative Execution (SE). SE reduces delay in a homogeneous environment by assigning delayed tasks to additional nodes. As cloud computing prevails, cloud computing environment is moving from homogeneous to heterogeneous. Original SE is not perfect and sometimes produces inefficient result in a heterogeneous environment. This paper proposes Dynamic Scheduling for Speculative Execution (DSSE) which enhances performance in a heterogeneous environment by improving existing SE. DSSE prevents wasted SE since it calculates processing capability of each node more objectively and precisely. DSSE has reduced entire processing time approximately 10% compared to original SE. Success rate of SE was 100%.

KW - Cloud Computing

KW - DSSE

KW - Heterogeneous environment

KW - MapReduce

KW - Speculative Execution

UR - http://www.scopus.com/inward/record.url?scp=84988329621&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84988329621&partnerID=8YFLogxK

U2 - 10.1109/ICDCSW.2014.23

DO - 10.1109/ICDCSW.2014.23

M3 - Conference contribution

AN - SCOPUS:84988329621

VL - 30-June-2014

SP - 119

EP - 124

BT - Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014

Y2 - 30 June 2014 through 3 July 2014

ER -

Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment

Abstract

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this