Have a personal or library account? Click to login
Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving Cover

Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving

Open Access
|Nov 2024

Abstract

We explore the use of radiance fields (RFs) to reconstruct photorealistic 3D urban scenes, creating digital twins (DTs) for autonomous driving (AD) by leveraging Nerfacto and Splatfacto models integrated with the CARLA simulator. Our research demonstrates that publicly available RFs can be utilized through Nerfstudio library to create photorealistic urban scenes and extract arbitrary images based on the camera pose. These scenes can serve as simulations for AD or as DT repositories for static environments within the vehicular metaverse. Additionally, we quantitatively evaluate RF models and use masking to remove dynamic objects, successfully simulating real-world scenarios. Quantitative evaluation shows that the Splatfacto model achieves a peak signal-to-noise ratio (PSNR) of up to 26.40, a structural similarity index measure (SSIM) of 0.84, and a learned perceptual image patch similarity (LPIPS) score of 0.21, consistently outperforming the Nerfacto model.

DOI: https://doi.org/10.2478/aei-2024-0015 | Journal eISSN: 1338-3957 | Journal ISSN: 1335-8243
Language: English
Page range: 27 - 34
Submitted on: Jun 24, 2024
Accepted on: Aug 26, 2024
Published on: Nov 17, 2024
Published by: Technical University of Košice
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year

© 2024 Matúš Dopiriak, Jakub Gerec, Juraj Gazda, published by Technical University of Košice
This work is licensed under the Creative Commons Attribution 4.0 License.