Three dimensional reconstruction of the cell cytoskeleton from stereo images

Abstract

Besides its primary application to robot vision, stereo vision also appears promising in the biomedical field. This study examines 3D reconstruction of the cell cytoskeleton. This application of stereo vision to electron micrographs extracts information about the interior structure of cells at the nanometer scale level. We propose two different types of stereo vision approaches: the line-segment and wavelet multiresolution methods. The former is primitive-based and the latter is a point-based approach. Structural information is stressed in both methods. Directional representation is employed to provide an ideal description for filament-type structures. In the line-segment method, line-segments are first extracted from directional representation and then matching is conducted between two line-segment sets of stereo images. A new search algorithm, matrix matching, is proposed to determine the matching globally. In the wavelet multiresolution method, a pyramidal architecture is presented. Bottom-up analysis is first performed to form two pyramids, containing wavelet decompositions and directional representations. Subsequently, top-down matching is carried out. Matching at a high level provides guidance and constraints to the matching at a lower level. Our reconstructed results reveal 3D structure and the relationships of filaments which are otherwise hard to see in the original stereo images. The method is sufficiently robust and accurate to allow the automated analysis of cell structural characteristics from electron microscopy pairs. The method may also have application to a general class of stereo images.