DNN-Based image classification for software GUI testing

Huijuan Lu; Li Wang; Minchao Ye; Ke Yan; Qun Jin

doi:10.1109/SmartWorld.2018.00306

DNN-Based image classification for software GUI testing

Huijuan Lu, Li Wang, Minchao Ye^*, Ke Yan, Qun Jin

^*Corresponding author for this work

School of Human Sciences

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

Since visual design of Graphical User Interface (GUI) is one of the indispensable parts of every software and to some extent, it determines whether the software is attractive. And with the increasing amount of GUI designs, it is usually impractical to deal with a variety of designs manually with limited staff. Therefore, a solution for that program is needed. This work designs and implements an automatic GUI testing system based on Deep Neural Network (DNN) and cloud service. The system includes screenshot capturing and uploading module, web server, DNN models, etc. It provides accurate and efficient GUI automatic testing technology.

Original language	English
Title of host publication	Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018
Editors	Frederic Loulergue, Guojun Wang, Md Zakirul Alam Bhuiyan, Xiaoxing Ma, Peng Li, Manuel Roveri, Qi Han, Lei Chen
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1818-1823
Number of pages	6
ISBN (Electronic)	9781538693803
DOIs	https://doi.org/10.1109/SmartWorld.2018.00306
Publication status	Published - 2018 Dec 4
Event	4th IEEE SmartWorld, 15th IEEE International Conference on Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018 - Guangzhou, China Duration: 2018 Oct 7 → 2018 Oct 11

Publication series

Name	Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018

Other

Other	4th IEEE SmartWorld, 15th IEEE International Conference on Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018
Country/Territory	China
City	Guangzhou
Period	18/10/7 → 18/10/11

Keywords

Cloud service
Deep neural network
GUI testing

ASJC Scopus subject areas

Computer Networks and Communications
Computer Science Applications
Information Systems and Management
Artificial Intelligence

Access to Document

10.1109/SmartWorld.2018.00306

Cite this

Lu, H., Wang, L., Ye, M., Yan, K., & Jin, Q. (2018). DNN-Based image classification for software GUI testing. In F. Loulergue, G. Wang, M. Z. A. Bhuiyan, X. Ma, P. Li, M. Roveri, Q. Han, & L. Chen (Eds.), Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018 (pp. 1818-1823). Article 8560284 (Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SmartWorld.2018.00306

DNN-Based image classification for software GUI testing. / Lu, Huijuan; Wang, Li; Ye, Minchao et al.
Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018. ed. / Frederic Loulergue; Guojun Wang; Md Zakirul Alam Bhuiyan; Xiaoxing Ma; Peng Li; Manuel Roveri; Qi Han; Lei Chen. Institute of Electrical and Electronics Engineers Inc., 2018. p. 1818-1823 8560284 (Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Lu, H, Wang, L, Ye, M, Yan, K & Jin, Q 2018, DNN-Based image classification for software GUI testing. in F Loulergue, G Wang, MZA Bhuiyan, X Ma, P Li, M Roveri, Q Han & L Chen (eds), Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018., 8560284, Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018, Institute of Electrical and Electronics Engineers Inc., pp. 1818-1823, 4th IEEE SmartWorld, 15th IEEE International Conference on Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018, Guangzhou, China, 18/10/7. https://doi.org/10.1109/SmartWorld.2018.00306

Lu H, Wang L, Ye M, Yan K, Jin Q. DNN-Based image classification for software GUI testing. In Loulergue F, Wang G, Bhuiyan MZA, Ma X, Li P, Roveri M, Han Q, Chen L, editors, Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 1818-1823. 8560284. (Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018). doi: 10.1109/SmartWorld.2018.00306

Lu, Huijuan ; Wang, Li ; Ye, Minchao et al. / DNN-Based image classification for software GUI testing. Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018. editor / Frederic Loulergue ; Guojun Wang ; Md Zakirul Alam Bhuiyan ; Xiaoxing Ma ; Peng Li ; Manuel Roveri ; Qi Han ; Lei Chen. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 1818-1823 (Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018).

@inproceedings{985cb361592048b491833c457ea139c4,

title = "DNN-Based image classification for software GUI testing",

abstract = "Since visual design of Graphical User Interface (GUI) is one of the indispensable parts of every software and to some extent, it determines whether the software is attractive. And with the increasing amount of GUI designs, it is usually impractical to deal with a variety of designs manually with limited staff. Therefore, a solution for that program is needed. This work designs and implements an automatic GUI testing system based on Deep Neural Network (DNN) and cloud service. The system includes screenshot capturing and uploading module, web server, DNN models, etc. It provides accurate and efficient GUI automatic testing technology.",

keywords = "Cloud service, Deep neural network, GUI testing",

author = "Huijuan Lu and Li Wang and Minchao Ye and Ke Yan and Qun Jin",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 4th IEEE SmartWorld, 15th IEEE International Conference on Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018 ; Conference date: 07-10-2018 Through 11-10-2018",

year = "2018",

month = dec,

day = "4",

doi = "10.1109/SmartWorld.2018.00306",

language = "English",

series = "Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1818--1823",

editor = "Frederic Loulergue and Guojun Wang and Bhuiyan, {Md Zakirul Alam} and Xiaoxing Ma and Peng Li and Manuel Roveri and Qi Han and Lei Chen",

booktitle = "Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018",

}

TY - GEN

T1 - DNN-Based image classification for software GUI testing

AU - Lu, Huijuan

AU - Wang, Li

AU - Ye, Minchao

AU - Yan, Ke

AU - Jin, Qun

PY - 2018/12/4

Y1 - 2018/12/4

N2 - Since visual design of Graphical User Interface (GUI) is one of the indispensable parts of every software and to some extent, it determines whether the software is attractive. And with the increasing amount of GUI designs, it is usually impractical to deal with a variety of designs manually with limited staff. Therefore, a solution for that program is needed. This work designs and implements an automatic GUI testing system based on Deep Neural Network (DNN) and cloud service. The system includes screenshot capturing and uploading module, web server, DNN models, etc. It provides accurate and efficient GUI automatic testing technology.

AB - Since visual design of Graphical User Interface (GUI) is one of the indispensable parts of every software and to some extent, it determines whether the software is attractive. And with the increasing amount of GUI designs, it is usually impractical to deal with a variety of designs manually with limited staff. Therefore, a solution for that program is needed. This work designs and implements an automatic GUI testing system based on Deep Neural Network (DNN) and cloud service. The system includes screenshot capturing and uploading module, web server, DNN models, etc. It provides accurate and efficient GUI automatic testing technology.

KW - Cloud service

KW - Deep neural network

KW - GUI testing

UR - http://www.scopus.com/inward/record.url?scp=85060304294&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060304294&partnerID=8YFLogxK

U2 - 10.1109/SmartWorld.2018.00306

DO - 10.1109/SmartWorld.2018.00306

M3 - Conference contribution

AN - SCOPUS:85060304294

T3 - Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018

SP - 1818

EP - 1823

BT - Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018

A2 - Loulergue, Frederic

A2 - Wang, Guojun

A2 - Bhuiyan, Md Zakirul Alam

A2 - Ma, Xiaoxing

A2 - Li, Peng

A2 - Roveri, Manuel

A2 - Han, Qi

A2 - Chen, Lei

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 4th IEEE SmartWorld, 15th IEEE International Conference on Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018

Y2 - 7 October 2018 through 11 October 2018

ER -

DNN-Based image classification for software GUI testing

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this