TY - GEN
T1 - When digital twin meets deep reinforcement learning in multi-UAV path planning
AU - Li, Siyuan
AU - Lin, Xi
AU - Wu, Jun
AU - Bashir, Ali Kashif
AU - Nawaz, Raheel
N1 - Publisher Copyright:
© 2022 ACM.
PY - 2022/10/17
Y1 - 2022/10/17
N2 - Unmanned aerial vehicles (UAVs) path planning is one of the promising technologies in the fifth-generation wireless communications. The gap between simulation and reality limits the application of deep reinforcement learning (DRL) in UAV path planning. Therefore, we propose a digital twin-based deep reinforcement learning training framework. With the help of digital twin, DRL model can be trained more effectively deployed to real UAVs. In this training framework, we propose a deep deterministic policy gradient (DDPG) based multi-UAV path planning algorithm. Based on decomposed actor structure in DRL, we design a pooling-based combined LSTM network to better understand different state information in a multi-UAV path planning task. Moreover, we also establish a digital twin platform for multi-UAV system, which has a high degree of simulation and visualization. The simulation result shows that the proposed algorithm can achieve higher mean rewards, and outperforms DDPG in average arrival rate by more than 30%.
AB - Unmanned aerial vehicles (UAVs) path planning is one of the promising technologies in the fifth-generation wireless communications. The gap between simulation and reality limits the application of deep reinforcement learning (DRL) in UAV path planning. Therefore, we propose a digital twin-based deep reinforcement learning training framework. With the help of digital twin, DRL model can be trained more effectively deployed to real UAVs. In this training framework, we propose a deep deterministic policy gradient (DDPG) based multi-UAV path planning algorithm. Based on decomposed actor structure in DRL, we design a pooling-based combined LSTM network to better understand different state information in a multi-UAV path planning task. Moreover, we also establish a digital twin platform for multi-UAV system, which has a high degree of simulation and visualization. The simulation result shows that the proposed algorithm can achieve higher mean rewards, and outperforms DDPG in average arrival rate by more than 30%.
KW - deep reinforcement learning
KW - digital twin
KW - flocking motion
KW - multi-UAV system
KW - path planning
UR - http://www.scopus.com/inward/record.url?scp=85141847418&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85141847418&partnerID=8YFLogxK
U2 - 10.1145/3555661.3560865
DO - 10.1145/3555661.3560865
M3 - Conference contribution
AN - SCOPUS:85141847418
T3 - DroneCom 2022 - Proceedings of the 5th International ACM Mobicom Workshop on Drone Assisted Wireless Communications for 5G and Beyond
SP - 61
EP - 66
BT - DroneCom 2022 - Proceedings of the 5th International ACM Mobicom Workshop on Drone Assisted Wireless Communications for 5G and Beyond
PB - Association for Computing Machinery, Inc
T2 - 5th International ACM Mobicom Workshop on Drone Assisted Wireless Communications for 5G and Beyond, DroneCom 2022
Y2 - 21 October 2022
ER -