RTSDM: A Real-Time Semantic Dense Mapping System for UAVs

RTSDM: A Real-Time Semantic Dense Mapping System for UAVs

Blog Article

Intelligent drones or flying here robots play a significant role in serving our society in applications such as rescue, inspection, agriculture, etc.Understanding the scene of the surroundings is an essential capability for further autonomous tasks.Intuitively, knowing the self-location of the UAV and creating a semantic 3D map is significant for fully autonomous tasks.However, integrating simultaneous localization, 3D reconstruction, and semantic segmentation together is a huge challenge for power-limited systems such as UAVs.To address this, we propose a real-time semantic mapping system that can help a power-limited UAV system to understand its location and surroundings.

The proposed approach includes a modified visual SLAM with the direct method to accelerate the computationally intensive feature matching process and a real-time semantic segmentation module at the back end.The semantic module runs a lightweight network, BiSeNetV2, and performs hobbit door for sale segmentation only at key frames from the front-end SLAM task.Considering fast navigation and the on-board memory resources, we provide a real-time dense-map-building module to generate an OctoMap with the segmented semantic map.The proposed system is verified in real-time experiments on a UAV platform with a Jetson TX2 as the computation unit.A frame rate of around 12 Hz, with a semantic segmentation accuracy of around 89% demonstrates that our proposed system is computationally efficient while providing sufficient information for fully autonomous tasks such as rescue, inspection, etc.