IEEE ICIP 2026

Constrained Dense Correspondence Graphs for Robust Structure-from-Motion Targeting Endoscopic Videos

Yu-Chun Lin1, Ming-Lun Han2, Kuang-Chen Yen2, Homer H. Chen1

1 National Taiwan University  ·  2 National Taiwan University Hospital

Abstract

Reconstructing complete 3D geometry from monocular clinical endoscopic videos is challenging due to weak texture, repetitive tissue patterns, and severe illumination artifacts. Although emerging dense matching methods exhibit improved resilience to textureless regions, they often produce abundant spurious correspondences across non-overlapping views, which corrupts the correspondence graph and causes structure-from-motion (SfM) pipelines to fail. We propose a framework for constructing a dense correspondence graph that leverages explicit temporal locality, parallax-driven geometric constraints, and loop-closure revisiting to enable reliable SfM for monocular endoscopic videos. Combined with illumination-aware masking and SfM initialization adapted to endoscopy, the method improves registration robustness and reconstruction completeness on phantom and clinical datasets.

Video preview

Drag to move · Preview

Interactive reconstructions

Drag to orbit, pinch or scroll to zoom. Drag the white divider to compare Ours (SfM point cloud) with dataset ground truth on C3VD, VR-CAPS, and EndoMapper (visible-surface crop of the rest-shape mesh; Seq 1–2 share that static GT and do not include tissue deformation).

Ours (SfM points)
Ground truth (dataset)
Loading…

Method compare

Side-by-side comparison of reconstructions on VR-CAPS sequences.

Left
Right
Loading…

Citation

@INPROCEEDINGS{11630223,
  author={Lin, Yu-Chun and Han, Ming-Lun and Yen, Kuang-Chen and Chen, Homer H.},
  booktitle={2026 IEEE International Conference on Image Processing (ICIP)},
  title={Constrained Dense Correspondence Graphs for Robust Structure-From-Motion Targeting Endoscopic Videos},
  year={2026},
  volume={},
  number={},
  pages={1-6},
  keywords={Printing;Three-dimensional displays;Videos;Sequential analysis;Sequences;Endoscopes;Geometry;Lighting;Biological tissues;Structure from motion;Endoscopy;3D reconstruction;structure-from-motion (SfM);feature matching;image registration},
  doi={10.1109/ICIP61757.2026.11630223}}

Acknowledgements

Supported by NTU (114L9009) and NSTC (113-2222-E-002-003-MY3). Clinical data collection was approved by the IRB of National Taiwan University Hospital (No. 202407132RINC). Interactive 3D on this page uses public phantom / simulated datasets only (C3VD, VR-CAPS, and EndoMapper simulated sequences).

Datasets and related work

Datasets

  1. T. L. Bobrow, M. Golhar, R. Vijayan, V. S. Akshintala, J. R. Garcia, and N. J. Durr, “Colonoscopy 3D video dataset with paired depth from 2D-3D registration,” Medical Image Analysis, vol. 90, p. 102956, 2023. (C3VD)
  2. K. Incetan, I. O. Celik, A. Obeid, N. I. Gokceler, K. Ozyoruk, Y. Almalioglu, R. J. Chen, N. I. Mahmood, H. Gilbert, N. J. Durr, and M. Turan, “VR-Caps: A virtual environment for capsule endoscopy,” Medical Image Analysis, vol. 70, p. 101990, 2021. (VR-CAPS)
  3. P. Azagra, C. Sostres, Á. Ferrández, L. Riazuelo, C. Tomasini, A. L. Barbed, J. Morlana, D. Recasens, V. M. Batlle, J. J. Gómez-Rodríguez, R. P. Pueyo, J. Civera, J. D. Tardós, A. C. Murillo, A. Lanas, and J. M. M. Montiel, “EndoMapper dataset of complete calibrated endoscopy procedures,” Scientific Data, vol. 10, 2023. (EndoMapper)

Methods cited in the paper

  1. V. Leroy, Y. Cabon, and J. Revaud, “Grounding image matching in 3D with MASt3R,” in ECCV, 2024.
  2. J. L. Schönberger and J.-M. Frahm, “Structure-from-motion revisited,” in CVPR, 2016. (COLMAP)
  3. Z. Teed and J. Deng, “RAFT: Recurrent all-pairs field transforms for optical flow,” in ECCV, 2020.
  4. G. Berton, C. Masone, and B. Caputo, “Rethinking visual geo-localization for large-scale applications,” in CVPR, 2022. (CosPlace)
  5. T.-Y. Wei, M.-L. Han, W.-C. Liao, K.-C. Yen, S.-J. Chen, and H. H. Chen, “Endoscopic feature enhancement for stomach 3D reconstruction without dyeing,” in ICIP, 2023.
  6. O. el Meslouhi, M. Kardouchi, H. Allali, T. Gadi, and Y. Ait Benkaddour, “Automatic detection and inpainting of specular reflections for colposcopic images,” Open Computer Science, 2011.
  7. V. M. Batlle, J. M. M. Montiel, P. Fua, and J. D. Tardós, “LightNeuS: Neural surface reconstruction in endoscopy using illumination decline,” in MICCAI, 2023.