Refining Ground Truth Poses in Autonomous Driving Datasets via Neural Rendering

IEEE RA-L 2026

Authors
Quentin Herau1,2Nathan Piasco1Moussab Bennehar1Luis Roldão1Dzmitry Tsishkou1Bingbing Liu1Cyrille Migniot3Pascal Vasseur4Cédric Demonceaux2

1 Noah’s Ark Lab, Huawei2 Université Bourgogne Europe, CNRS, ICB UMR 6303, Dijon, France3 Université Bourgogne Europe, ImViA UR 7535, Dijon, France4 MIS UR 4290, Université de Picardie Jules Verne, Amiens, France

HuaweiICBImViAMISCNRSUniversité Bourgogne EuropeUniversité de Picardie Jules Verne
PaperPublisherLeaderboardCalibration codeBenchmark code

Refine sensor calibration and vehicle motion, and evaluate pose quality through rendering, structure-from-motion and geometric consistency.

Pose refinement and evaluation

AVoptim evaluates the consequences of sensor calibration and trajectory refinement on driving-scene reconstruction. MOISST++ extends scene-based calibration to ego motion. The benchmark reports calibration and trajectory refinement on four datasets, averaged over five sequences each.

The published pipeline prepares observations, refines sensor extrinsics, then refines trajectories and evaluates the resulting poses.
The published pipeline prepares observations, refines sensor extrinsics, then refines trajectories and evaluates the resulting poses. Paper figure ↗

Visual comparisons

Renderings from the original and refined poses, using the project’s comparison videos and images.

nuScenes

nuScenes reconstructed with refined posesnuScenes reconstructed with original posesOriginalRefined
Drag the divider or use the arrow keys.

Waymo

Waymo reconstructed with refined posesWaymo reconstructed with original posesOriginalRefined
Drag the divider or use the arrow keys.

PandaSet · street

PandaSet · street reconstructed with refined posesPandaSet · street reconstructed with original posesOriginalRefined
Drag the divider or use the arrow keys.

PandaSet · road markings

PandaSet · road markings reconstructed with refined posesPandaSet · road markings reconstructed with original posesOriginalRefined
Drag the divider or use the arrow keys.

Leaderboard

Calibration + trajectory refinement. Original and MOISST++ are compared across all ten pose-benchmark scores: rendering quality, structure from motion and geometric consistency.

Dataset averages over five sequences; overall averages over all twenty. Best scores in each metric are bold.
DatasetPose setNerfactoSplatfactoSfMGeometry
PSNR
dB ↑		SSIM
↑		LPIPS
↓		PSNR
dB ↑		SSIM
↑		LPIPS
↓		Reprojection
px ↓		Track length
↑		P2M precision
% ↑		Surface distance
m ↓
All datasetsMean over 20 sequencesOriginalDataset poses21.800.7200.45523.110.7720.3331.1086.1154.980.296
All datasetsMean over 20 sequencesMOISST++Calibration + trajectory22.690.7600.38725.030.8200.2660.8866.9568.740.204

Download scores CSVDownload scores JSONScore definitions and aggregation

Evaluation protocol

Each dataset contributes five sequences. Dataset scores come from reference/current/*_pose.csv in the benchmark repository. Overall scores are the arithmetic mean of the unrounded twenty-sequence records for each pose set.

MOISST++ refines sensor extrinsics and ego trajectory; Original is generated locally from the downloaded dataset poses. Rendering evaluation uses Nerfacto with LiDAR depth supervision and Splatfacto with SfM + LiDAR initialization, with pose optimization disabled. PSNR, SSIM and LPIPS are part of the pose benchmark.

SfM triangulates with the supplied camera poses and intrinsics fixed. P2M precision uses a 0.15 m nearest-vertex threshold; surface distance uses triangle surfaces. These scores measure consistency rather than independent absolute pose ground truth.

Research code

neural-calibration prepares raw KITTI-360, nuScenes, PandaSet and Waymo data for independent MOISST, SOAC, 3DGS-Calib, MOISST++ and SOAC++ implementations, with training and sensor-pose export instructions. avoptim-benchmark prepares the twenty benchmark sequences and evaluates Original, provided MOISST++ or custom method poses across ten rendering, structure-from-motion and geometry metrics. Original poses and observation metadata are generated locally from authorized dataset downloads; the provided optimized bundle contains pose matrices only. Training and benchmark preparation use separate commands documented in the repositories.

The results shown here are reported paper results and recorded benchmark scores. See the repositories for implementation details and validation scope.

Citation

BibTeX
@article{herau2026refining,
  title = {Refining Ground Truth Poses in Autonomous Driving Datasets via Neural Rendering},
  author = {Herau, Quentin and Piasco, Nathan and Bennehar, Moussab and Roldao, Luis and Tsishkou, Dzmitry and Liu, Bingbing and Migniot, Cyrille and Vasseur, Pascal and Demonceaux, C\'edric},
  journal = {IEEE Robotics and Automation Letters},
  year = {2026},
  doi = {10.1109/LRA.2026.3732898},
  eprint = {2504.15776},
  archivePrefix = {arXiv}
}

Download BibTeX

Paper figure