LuSNAR: A Lunar Segmentation, Navigation, and Reconstruction Dataset based on Multi-sensor for Autonomous Exploration

Subsequently, a stereo camera, LiDAR, and IMU were configured in AirSim. After launching Unreal Engine and AirSim, the lunar rover was manually controlled to move along a predefined path, with motion modes including straight-line movement, turning, climbing, descending, obstacle avoidance, and circumnavigation. The AirSim API was used to record the rover's 6D pose ground truth and IMU data in real time.

Then, using the recorded poses and sensor extrinsics, stereo images, depth maps, and semantic images were rendered offline, and LiDAR point clouds were generated. Finally, all data were saved by timestamp. Each scene covers approximately \(300 \times 300\) m, the rover travel distance is about 200–600 m, and the speed does not exceed \(5\) m/s.

The sequence length of each scene is about 1000–2000 frames. The nine sequences collectively form about 13,006 frames of images and 13,006 frames of point clouds for experiments such as semantic segmentation. The total size of the dataset is about 108 GB, including over 42 GB of stereo image pairs, about 50 GB of depth maps, about 356 MB of 2D semantic labels, and about 14 GB of single-frame 3D semantic point clouds.In terms of sensors and spatiotemporal resolution, the dataset contains one stereo camera, one spinning LiDAR, and one IMU.

The stereo camera has a capture frequency of \(10\) Hz, a resolution of \(1024 \times 1024\), a focal length of \(610.17784\), a field of view of \(80^\circ \times 80^\circ\), and a baseline of \(310\) mm. The LiDAR has 128 beams, \(10\) Hz, a horizontal field of view of \(360^\circ\), a vertical field of view from \(-25^\circ\) to \(27^\circ\), a maximum range of no more than \(30\) m, and a point rate of up to 20 million points per second.

The IMU operates at \(100\) Hz, is 3-axis, and is configured with noise parameters such as accelerometer and gyroscope random walk and bias instability. Data files are named by timestamp. Each sequence includes: \(1024 \times 1024\) PNG-format RGB images, \(1024 \times 1024\) PNG-format 2D semantic segmentation labels (different colors represent different objects), \(1024 \times 1024\) 16-bit PFM-format depth maps, TXT-format 3D semantic point clouds on the order of \(10^4\) (different IDs represent different categories), TXT-format IMU data, TXT-format 6D lunar rover pose ground truth in the world coordinate system, and timestamp files for the camera, LiDAR, and IMU.

The 2D semantic categories are lunar regolith, rock, impact crater, mountain, and sky; the 3D semantic point cloud categories are lunar regolith, rock, and impact crater. The temporal resolution is \(10\) Hz for the camera, \(10\) Hz for LiDAR, and \(100\) Hz for IMU. In terms of spatial resolution, the images and semantic labels are \(1024 \times 1024\), the depth maps are \(1024 \times 1024\) 16-bit data, and the LiDAR has 128 beams with a maximum range of \(30\) m.

In terms of spatial extent, each scene is about \(300\) m \(\times\) \(300\) m, and the trajectory length is about 200–600 m.Regarding data missingness and errors, the paper states that 5% of the generated data from the nine scenes were randomly sampled for semi-automatic verification, totaling 650 synchronized samples. Timestamp consistency, cross-modal alignment, and overall consistency were checked. All samples passed, with an overall consistency of \(100\%\).

No missing timestamps, duplicate timestamps, or modality-level synchronization failures were found, nor were obvious cross-modal misalignments or severe label inconsistencies observed. However, a domain gap still exists between the simulated data and the real lunar environment: the camera uses an ideal pinhole model and does not include lens distortion, vignetting, or thermal noise; LiDAR is based on geometric raycasting and simplifies multi-echo returns and material reflectivity; although the IMU includes white noise and bias instability, it is still modeled noise; complex physical interactions such as lunar dust accumulation, dynamic light scattering, and wheel sinkage are not simulated.

Error sources also include long-range quantization error in the 16-bit depth, aliasing caused by sparse LiDAR point clouds at long range, and possible motion blur in high-speed scenarios due to \(10\) Hz camera sampling. Semantic labels are generated directly by the simulation engine's G-Buffer, and are therefore pixel-perfect, without human subjective annotation error.