Follow:

Three-dimensional reconstruction of highly complex microscopic samples using scanning electron microscopy and optical flow estimation

https://doi.org/10.1371/journal.pone.0175078

“Scanning Electron Microscope (SEM) as one of the major research and industrial equipment for imaging of micro-scale samples and surfaces has gained extensive attention from its emerge. However, the acquired micrographs still remain two-dimensional (2D). In the current work a novel and highly accurate approach is proposed to recover the hidden third-dimension by use of multi-view image acquisition of the microscopic samples combined with pre/post-processing steps including sparse feature-based stereo rectification, nonlocal-based optical flow estimation for dense matching and finally depth estimation. Employing the proposed approach, three-dimensional (3D) reconstructions of highly complex microscopic samples were achieved to facilitate the interpretation of topology and geometry of surface/shape attributes of the samples. As a byproduct of the proposed approach, high-definition 3D printed models of the samples can be generated as a tangible means of physical understanding. Extensive comparisons with the state-of-the-art reveal the strength and superiority of the proposed method in uncovering the details of the highly complex microscopic samples.”

“Scanning electron microscope (SEM) is one of the principal research and industrial equipment for imaging on the microscopic scale. SEM and its diverse applications have been a very active research area over the recent decades, and scientific studies well covered the use of SEM in broad domains ranging from biomedical applications to materials sciences and nano technologies [17]. SEM as an advanced microscopy device produces high quality images of microscopic specimen using a focused beam of electrons which can be then captured by two types of detectors, secondary electron (SE) and back-scattered electron (BSE) detectors, to provide both compositional and/or geometrical information about the microscopic surface [8]. However, SEM micrographs remain 2D while the need for having a more quantitative knowledge of the 3D surface of the microscopic samples is of high importance. Serial section transmission electron microscopy (ssTEM) [9], serial blockface SEM (SBF-SEM) [1011] and focused ion beam SEM (FIB-SEM) [1213] are among the widely-used volume electron microscopy devices. While many initially developed as means of imaging of the brain tissues, examples of usage for other biological tissues have been reported in the literature [14]. The procedure of imaging using such devices generally involves sectioning of ultra-thin layers of the tissue and then imaging in order to be able to build a full volume model of the tissue. Sectioning is performed manually in ssTEM while the procedure is done automatically in SBF-SEM (using a diamond knife) and FIB-SEM (using focused gallium ion beam). Using such devices it is possible to acquire high-resolution volume scans of the biological samples. However, due the destructive nature of such imaging procedures, the samples cannot be revisited. Image alignment, rotational errors and charging artifacts may compromise interpretation of volume EM data. The remedies can be sough in specific procedures for specimen preparation or pre-processing steps of image registration. Moreover, segmentation of the features of interest for 3D model reconstruction imposes additional challenges for proper interpretation of such data, especially for the problem of surface assessment since very fine details can be eliminated due to various sources of error mentioned above. These limitations make volume EM imaging not suitable for accurate surface reconstruction of microscopic samples.”

SEM imaging protocol

In this work, a Hitachi S-4800 field emission scanning electron microscope (FE-SEM) has been utilized to generate the micrographs. This SEM is equipped with a computer controlled 5 axis motorized specimen stage which enables movements in xy and z directions as well as tilt (-5 to 70°) and rotation (0 to 360°). Specimen manipulations, such as tilt, z-positioning and rotation of the specimen stage, as well as image pre-processing and capture functions were operated through the Hitachi PC-SEM software. The working distance which gives the required depth of focus was determined at the maximum tilt for every single sample at the magnification chosen for image capture. As the specimen was tilted in successive 1° increments until reaching the final value through the software application, the SEM image was centered by moving the stage in the x– and/or y-axes manually. The micrographs were acquired with an accelerating voltage of 3 or 5 kV, utilizing the signals from both the upper and lower SE detectors in a mixed manner, as shown in Fig 1. The magnification and working distance were held fixed in each captured image of the tilt series. Contrast and brightness were adjusted manually to keep consistency between SEM micrographs. Fig 2 summarizes the data that used in this work. Micrographs from Arabidopsis Anther 1Arabidopsis Anther 2GraphenePseudoscorpion and Fly Ash are considered for evaluating the performance and accuracy of the proposed approach.

thumbnail

Fig 1. SEM imaging procedure used for this study.

https://doi.org/10.1371/journal.pone.0175078.g001

thumbnail

Fig 2. Dataset acquired using a Hitachi S-4800 Field Emission Scanning Electron Microscope (FE-SEM) by tilting the specimen stage by 7°.
The samples are (a) Arabidopsis Anther 1 (1280 × 960), (b) Arabidopsis Anther 2 (1280 × 960), (c) Graphene (1280 × 960), (d) Pseudoscorpion (960 × 1280) and (e) Fly Ash (926 × 924). The micrographs for the Pseudoscorpion set are rotated by 90° for visualization purposes.

https://doi.org/10.1371/journal.pone.0175078.g002

Leave a Comment