Learning-based algorithms with application to urban scene autonomous driving

Urban roads are one of the most complicated applications in autonomous driving. The main bottleneck lies in perception and decision-making algorithms. In this work, we propose a new learning-based autonomous driving system, including a novel Convolutional Neural Network (CNN)-based multi-sensor fusion object detector, and a novel Deep Reinforcement Learning (DRL)-based decision planner. Multi-sensor fusion object detector integrates two advanced CNN-based object detectors to separately detect objects from camera image and LiDAR point cloud with high precision and processing speed. Meanwhile, a stereo vision integrated Camera-LiDAR object fusion method is proposed to complementarily fuse two sensor detections. Besides, a DRL-based decision planner is proposed by integrating DRL-based tactical long-term decision-making and spatiotemporal short-term trajectory planning in dynamic urban driving scenarios with efficiency, safety and comfort. Finally, we train the algorithms and do joint testing in real scenarios. The experimental results show that the proposed system could meet the requirements of autonomous driving in urban scene.