A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing

Wei Liang; Yonghua Xiong; Min Wu; Qun Jin

doi:10.1109/SmartCity.2015.190

A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing

Wei Liang, Yonghua Xiong, Min Wu, Qun Jin

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

Data intensive computing (DIC) offers an attractive option for business to remotely execute applications and load the computing resources from cloud in a streaming way. A key challenge in such environment is to increase the utilization of cloud cluster for the high throughput processing. One way of achieving this goal is to optimize the execution of computing jobs on the cluster. We observe that the order in which these jobs are executed can have a significant impact on their overall completion time (makespan). Our goal is to design a job scheduler that minimizes the makespan. In this study, a new formalization is introduced to present each job as a pair of disk processing and network transmitting two-stage durations. Due to the streaming processing feature, the two-stage operations are executed in an overlap manner and may lead to both one-stage and two-stage scheduling situations. A novel heuristic scheduling strategy is proposed for this hybrid scheduling problem, and the performance of the method is confirmed by the experimental evaluation.

Original language	English
Title of host publication	Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	932-937
Number of pages	6
ISBN (Electronic)	9781509018932
DOIs	https://doi.org/10.1109/SmartCity.2015.190
Publication status	Published - 2016 May 2
Event	IEEE International Conference on Smart City, SmartCity 2015 - Chengdu, China Duration: 2015 Dec 19 → 2015 Dec 21

Other

Other	IEEE International Conference on Smart City, SmartCity 2015
Country/Territory	China
City	Chengdu
Period	15/12/19 → 15/12/21

Keywords

Data Intensive Computing
Makespan
NP
Scheduling

ASJC Scopus subject areas

Information Systems
Media Technology
Computer Science Applications
Signal Processing
Computer Networks and Communications
Modelling and Simulation
Sociology and Political Science
Urban Studies

Access to Document

10.1109/SmartCity.2015.190

Cite this

Liang, W., Xiong, Y., Wu, M., & Jin, Q. (2016). A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing. In Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015 (pp. 932-937). [7463845] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SmartCity.2015.190

A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing. / Liang, Wei; Xiong, Yonghua; Wu, Min et al.
Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015. Institute of Electrical and Electronics Engineers Inc., 2016. p. 932-937 7463845.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Liang, W, Xiong, Y, Wu, M & Jin, Q 2016, A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing. in Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015., 7463845, Institute of Electrical and Electronics Engineers Inc., pp. 932-937, IEEE International Conference on Smart City, SmartCity 2015, Chengdu, China, 15/12/19. https://doi.org/10.1109/SmartCity.2015.190

Liang W, Xiong Y, Wu M, Jin Q. A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing. In Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015. Institute of Electrical and Electronics Engineers Inc. 2016. p. 932-937. 7463845 doi: 10.1109/SmartCity.2015.190

Liang, Wei ; Xiong, Yonghua ; Wu, Min et al. / A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing. Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 932-937

@inproceedings{645c74f1e8fe4f05a1b5b8ea8fbf64ab,

title = "A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing",

abstract = "Data intensive computing (DIC) offers an attractive option for business to remotely execute applications and load the computing resources from cloud in a streaming way. A key challenge in such environment is to increase the utilization of cloud cluster for the high throughput processing. One way of achieving this goal is to optimize the execution of computing jobs on the cluster. We observe that the order in which these jobs are executed can have a significant impact on their overall completion time (makespan). Our goal is to design a job scheduler that minimizes the makespan. In this study, a new formalization is introduced to present each job as a pair of disk processing and network transmitting two-stage durations. Due to the streaming processing feature, the two-stage operations are executed in an overlap manner and may lead to both one-stage and two-stage scheduling situations. A novel heuristic scheduling strategy is proposed for this hybrid scheduling problem, and the performance of the method is confirmed by the experimental evaluation.",

keywords = "Data Intensive Computing, Makespan, NP, Scheduling",

author = "Wei Liang and Yonghua Xiong and Min Wu and Qun Jin",

year = "2016",

month = may,

day = "2",

doi = "10.1109/SmartCity.2015.190",

language = "English",

pages = "932--937",

booktitle = "Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "IEEE International Conference on Smart City, SmartCity 2015 ; Conference date: 19-12-2015 Through 21-12-2015",

}

TY - GEN

T1 - A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing

AU - Liang, Wei

AU - Xiong, Yonghua

AU - Wu, Min

AU - Jin, Qun

PY - 2016/5/2

Y1 - 2016/5/2

N2 - Data intensive computing (DIC) offers an attractive option for business to remotely execute applications and load the computing resources from cloud in a streaming way. A key challenge in such environment is to increase the utilization of cloud cluster for the high throughput processing. One way of achieving this goal is to optimize the execution of computing jobs on the cluster. We observe that the order in which these jobs are executed can have a significant impact on their overall completion time (makespan). Our goal is to design a job scheduler that minimizes the makespan. In this study, a new formalization is introduced to present each job as a pair of disk processing and network transmitting two-stage durations. Due to the streaming processing feature, the two-stage operations are executed in an overlap manner and may lead to both one-stage and two-stage scheduling situations. A novel heuristic scheduling strategy is proposed for this hybrid scheduling problem, and the performance of the method is confirmed by the experimental evaluation.

AB - Data intensive computing (DIC) offers an attractive option for business to remotely execute applications and load the computing resources from cloud in a streaming way. A key challenge in such environment is to increase the utilization of cloud cluster for the high throughput processing. One way of achieving this goal is to optimize the execution of computing jobs on the cluster. We observe that the order in which these jobs are executed can have a significant impact on their overall completion time (makespan). Our goal is to design a job scheduler that minimizes the makespan. In this study, a new formalization is introduced to present each job as a pair of disk processing and network transmitting two-stage durations. Due to the streaming processing feature, the two-stage operations are executed in an overlap manner and may lead to both one-stage and two-stage scheduling situations. A novel heuristic scheduling strategy is proposed for this hybrid scheduling problem, and the performance of the method is confirmed by the experimental evaluation.

KW - Data Intensive Computing

KW - Makespan

KW - NP

KW - Scheduling

UR - http://www.scopus.com/inward/record.url?scp=84973889235&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84973889235&partnerID=8YFLogxK

U2 - 10.1109/SmartCity.2015.190

DO - 10.1109/SmartCity.2015.190

M3 - Conference contribution

AN - SCOPUS:84973889235

SP - 932

EP - 937

BT - Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - IEEE International Conference on Smart City, SmartCity 2015

Y2 - 19 December 2015 through 21 December 2015

ER -

A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing

Abstract

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this