Augmented reality shopping system through image search and virtual shop generation

Zhinan Li; Ruichen Ma; Kohei Obuchi; Boyang Liu; Kelvin Cheng; Soh Masuko; Jiro Tanaka

doi:10.1007/978-3-030-50020-7_26

Augmented reality shopping system through image search and virtual shop generation

Zhinan Li^*, Ruichen Ma, Kohei Obuchi, Boyang Liu, Kelvin Cheng, Soh Masuko, Jiro Tanaka

^*Corresponding author for this work

Graduate School of Information, Production and Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

Abstract

In this paper, we introduce an intelligent augmented reality shopping system that supports image search and 3D virtual shop generation. By capturing an image of the product that the user is looking at, and using image search, we can recommend similar products to the user. Based on the spatial understanding techniques, we developed a 3D virtual shop interface that would be automatically generated in the user’s physical room, where products are arranged around the room at appropriate locations. With the virtual shop display, users would be able to interact with the products’ life-size virtual models by full-hand manipulation. Our system provides more natural interactions such as grabbing and touching. We also carried out an evaluation study for using our system, the participants’ feedback shows that our system provides a better shopping experience with its intuitive search process, immersive shop display, and natural interactions.

Original language	English
Title of host publication	Human Interface and the Management of Information. Designing Information - Thematic Area, HIMI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings
Editors	Sakae Yamamoto, Hirohiko Mori
Publisher	Springer
Pages	363-376
Number of pages	14
ISBN (Print)	9783030500191
DOIs	https://doi.org/10.1007/978-3-030-50020-7_26
Publication status	Published - 2020
Event	Thematic Area on Human Interface and the Management of Information, HIMI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020 - Copenhagen, Denmark Duration: 2020 Jul 19 → 2020 Jul 24

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12184 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Thematic Area on Human Interface and the Management of Information, HIMI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020
Country/Territory	Denmark
City	Copenhagen
Period	20/7/19 → 20/7/24

Keywords

Full-hand manipulation
Image search
Immersive shopping
Intelligent system
Spatial understanding

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-50020-7_26

Cite this

Li, Z., Ma, R., Obuchi, K., Liu, B., Cheng, K., Masuko, S., & Tanaka, J. (2020). Augmented reality shopping system through image search and virtual shop generation. In S. Yamamoto, & H. Mori (Eds.), Human Interface and the Management of Information. Designing Information - Thematic Area, HIMI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings (pp. 363-376). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12184 LNCS). Springer. https://doi.org/10.1007/978-3-030-50020-7_26

Augmented reality shopping system through image search and virtual shop generation. / Li, Zhinan; Ma, Ruichen; Obuchi, Kohei et al.
Human Interface and the Management of Information. Designing Information - Thematic Area, HIMI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings. ed. / Sakae Yamamoto; Hirohiko Mori. Springer, 2020. p. 363-376 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12184 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Li, Z, Ma, R, Obuchi, K, Liu, B, Cheng, K, Masuko, S & Tanaka, J 2020, Augmented reality shopping system through image search and virtual shop generation. in S Yamamoto & H Mori (eds), Human Interface and the Management of Information. Designing Information - Thematic Area, HIMI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12184 LNCS, Springer, pp. 363-376, Thematic Area on Human Interface and the Management of Information, HIMI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020, Copenhagen, Denmark, 20/7/19. https://doi.org/10.1007/978-3-030-50020-7_26

Li Z, Ma R, Obuchi K, Liu B, Cheng K, Masuko S et al. Augmented reality shopping system through image search and virtual shop generation. In Yamamoto S, Mori H, editors, Human Interface and the Management of Information. Designing Information - Thematic Area, HIMI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings. Springer. 2020. p. 363-376. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-50020-7_26

Li, Zhinan ; Ma, Ruichen ; Obuchi, Kohei et al. / Augmented reality shopping system through image search and virtual shop generation. Human Interface and the Management of Information. Designing Information - Thematic Area, HIMI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings. editor / Sakae Yamamoto ; Hirohiko Mori. Springer, 2020. pp. 363-376 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{0f446f0ed10245cfbd2f17f5fc68276d,

title = "Augmented reality shopping system through image search and virtual shop generation",

abstract = "In this paper, we introduce an intelligent augmented reality shopping system that supports image search and 3D virtual shop generation. By capturing an image of the product that the user is looking at, and using image search, we can recommend similar products to the user. Based on the spatial understanding techniques, we developed a 3D virtual shop interface that would be automatically generated in the user{\textquoteright}s physical room, where products are arranged around the room at appropriate locations. With the virtual shop display, users would be able to interact with the products{\textquoteright} life-size virtual models by full-hand manipulation. Our system provides more natural interactions such as grabbing and touching. We also carried out an evaluation study for using our system, the participants{\textquoteright} feedback shows that our system provides a better shopping experience with its intuitive search process, immersive shop display, and natural interactions.",

keywords = "Full-hand manipulation, Image search, Immersive shopping, Intelligent system, Spatial understanding",

author = "Zhinan Li and Ruichen Ma and Kohei Obuchi and Boyang Liu and Kelvin Cheng and Soh Masuko and Jiro Tanaka",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2020.; Thematic Area on Human Interface and the Management of Information, HIMI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020 ; Conference date: 19-07-2020 Through 24-07-2020",

year = "2020",

doi = "10.1007/978-3-030-50020-7_26",

language = "English",

isbn = "9783030500191",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "363--376",

editor = "Sakae Yamamoto and Hirohiko Mori",

booktitle = "Human Interface and the Management of Information. Designing Information - Thematic Area, HIMI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings",

}

TY - GEN

T1 - Augmented reality shopping system through image search and virtual shop generation

AU - Li, Zhinan

AU - Ma, Ruichen

AU - Obuchi, Kohei

AU - Liu, Boyang

AU - Cheng, Kelvin

AU - Masuko, Soh

AU - Tanaka, Jiro

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2020.

PY - 2020

Y1 - 2020

N2 - In this paper, we introduce an intelligent augmented reality shopping system that supports image search and 3D virtual shop generation. By capturing an image of the product that the user is looking at, and using image search, we can recommend similar products to the user. Based on the spatial understanding techniques, we developed a 3D virtual shop interface that would be automatically generated in the user’s physical room, where products are arranged around the room at appropriate locations. With the virtual shop display, users would be able to interact with the products’ life-size virtual models by full-hand manipulation. Our system provides more natural interactions such as grabbing and touching. We also carried out an evaluation study for using our system, the participants’ feedback shows that our system provides a better shopping experience with its intuitive search process, immersive shop display, and natural interactions.

AB - In this paper, we introduce an intelligent augmented reality shopping system that supports image search and 3D virtual shop generation. By capturing an image of the product that the user is looking at, and using image search, we can recommend similar products to the user. Based on the spatial understanding techniques, we developed a 3D virtual shop interface that would be automatically generated in the user’s physical room, where products are arranged around the room at appropriate locations. With the virtual shop display, users would be able to interact with the products’ life-size virtual models by full-hand manipulation. Our system provides more natural interactions such as grabbing and touching. We also carried out an evaluation study for using our system, the participants’ feedback shows that our system provides a better shopping experience with its intuitive search process, immersive shop display, and natural interactions.

KW - Full-hand manipulation

KW - Image search

KW - Immersive shopping

KW - Intelligent system

KW - Spatial understanding

UR - http://www.scopus.com/inward/record.url?scp=85088744545&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85088744545&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-50020-7_26

DO - 10.1007/978-3-030-50020-7_26

M3 - Conference contribution

AN - SCOPUS:85088744545

SN - 9783030500191

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 363

EP - 376

BT - Human Interface and the Management of Information. Designing Information - Thematic Area, HIMI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings

A2 - Yamamoto, Sakae

A2 - Mori, Hirohiko

PB - Springer

T2 - Thematic Area on Human Interface and the Management of Information, HIMI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020

Y2 - 19 July 2020 through 24 July 2020

ER -

Augmented reality shopping system through image search and virtual shop generation

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this