Deep-Reinforcement-Learning-Based Cybertwin Architecture for 6G IIoT: An Integrated Design of Control, Communication, and Computing

Hansong Xu, Jun Wu*, Jianhua Li, Xi Lin*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

20 Citations (Scopus)


The cybertwin and 6G-enabled Industrial Internet of Things (6G-IIoT) are the critical technologies that create the digital counterparts for physical systems and enable the near-instant interconnectivity in the industrial domain. It is in demand but challenging to conduct the integrated design for 6G-IIoT, which intertwines the cyber subsystems, such as control, communication, computing (3C), and the physical industrial factories and plants. Therefore, the cybertwin, which synchronizes between the digital counterparts and its physical entities during the system runtime, is the ideal proving ground for conducting the integrated design on the highly intertwined 3C of 6G-IIoT. However, the cybertwin lacks artificial intelligence to capacitate the automated integrated design for the 6G-IIoT. In this article, we first demonstrate the architecture of the machine-learning-based cybertwin for 6G-IIoT. Then, we leverage deep reinforcement learning (DRL) to conduct the integrated design via systematic trial and error in the cybertwin model, which is otherwise costly and dangerous in real industrial systems. Moreover, we invent the adaptive observation window for deep Q -network (AOW-DQN), which generates system states adaptive to the control system's physical dynamics. Finally, the experimental results demonstrate the effectiveness and efficiency of our approach. To the best of our knowledge, we are the first to present the machine-learning-based cybertwin for carrying out the integrated design on the 3C for 6G-IIoT.

Original languageEnglish
Pages (from-to)16337-16348
Number of pages12
JournalIEEE Internet of Things Journal
Issue number22
Publication statusPublished - 2021 Nov 15
Externally publishedYes


  • 6G Industrial Internet of Things (6G-IIoT)
  • cybertwin
  • deep reinforcement learning (DRL)
  • integrated design

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications


Dive into the research topics of 'Deep-Reinforcement-Learning-Based Cybertwin Architecture for 6G IIoT: An Integrated Design of Control, Communication, and Computing'. Together they form a unique fingerprint.

Cite this