Improving 3D surface reconstruction from endoscopic video via fusion and refined reflectance modeling

Rui Wang; True Price; Qingyu Zhao; Jan-Michael Frahm; Julian Rosenman; Stephen Pizer

doi:10.1117/12.2254587

24 February 2017 Improving 3D surface reconstruction from endoscopic video via fusion and refined reflectance modeling

Rui Wang, True Price, Qingyu Zhao, Jan-Michael Frahm, Julian Rosenman, Stephen Pizer

Proceedings Volume 10133, Medical Imaging 2017: Image Processing; 101330B (2017) https://doi.org/10.1117/12.2254587
Event: SPIE Medical Imaging, 2017, Orlando, Florida, United States

Abstract

Shape from shading (SFS) has been studied for decades; nevertheless, its overly simple assumptions and its ill-conditioning have resulted in infrequent use in real applications. Price et al. recently developed an iterative scheme named shape from motion and shading (SFMS) that models both shape and reflectance of an unknown surface simultaneously. SFMS produces a fairly accurate, dense 3D reconstruction from each frame of a pharyngeal endoscopic video, albeit with inconsistency between the 3D reconstructions of different frames. We present a comprehensive study of the SFMS scheme and several improvements to it: (1) We integrate a deformable registration method into the iterative scheme and use the fusion of multiple surfaces as a reference surface to guide the next iteration’s reconstruction. This can be interpreted as incorporating regularity of a frame’s reconstruction with that of temporally nearby frames. (2) We show that the reflectance model estimation is crucial and very sensitive to noise in the data. Moreover, even when the surface reflection is not assumed to be Lambertian, the reflectance model estimation function in SFMS is still overly simple for endoscopy of human tissue. By removing outlier pixels, by preventing unrealistic BRDF estimation, and by reducing the falloff speed of illumination in SFS to account for the effect of multiple bouncing of the light, we improve the reconstruction accuracy.

Conference Presentation

Citation Download Citation

Rui Wang, True Price, Qingyu Zhao, Jan-Michael Frahm, Julian Rosenman, and Stephen Pizer "Improving 3D surface reconstruction from endoscopic video via fusion and refined reflectance modeling", Proc. SPIE 10133, Medical Imaging 2017: Image Processing, 101330B (24 February 2017); https://doi.org/10.1117/12.2254587

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available