Three-dimensional color photon counting microscopy using Bayesian estimation with adaptive priori information

Myungjin Cho

doi:10.3788/COL201513.070301

Abstract

In this Letter, we propose a novel three-dimensional (3D) color microscopy for microorganisms under photon-starved conditions using photon counting integral imaging and Bayesian estimation with adaptive priori information. In photon counting integral imaging, 3D images can be visualized using maximum likelihood estimation (MLE). However, since MLE does not consider a priori information of objects, the visual quality of 3D images may not be accurate. In addition, the only grayscale image can be reconstructed. Therefore, to enhance the visual quality of 3D images, we propose photon counting microscopy using maximum a posteriori with adaptive priori information. In addition, we consider a wavelength of each basic color channel to reconstruct 3D color images. To verify our proposed method, we carry out optical experiments.

Under photon-starved conditions such as low light level environment, three-dimensional (3D) image sensing and visualization for microorganisms have recently become a challenging topic. In conventional microscopy, since illumination devices with high-power radiation may be used to detect the image of microorganisms, it may cause damage, deformation, or destruction of the structure of 3D microorganisms. To avoid this problem, light sources with low-power radiation may be required. A photon counting detector or imaging technique^[1–5] may be applied to this microscopy system because there are few photons in low-power radiation. To reconstruct or visualize 3D images for microorganisms, integral imaging^[6–13] may be used which can obtain 3D information by capturing multiple 2D images with different perspectives through a lenslet array or camera array. In photon counting integral imaging, statistical estimation methods such as maximum likelihood estimation (MLE)^[3] or maximum a posteriori (MAP) with fixed priori information^[4] can be used for 3D visualization. However, these estimation methods have limitations. In MLE, since priori information is assumed to be a uniform distribution, estimation results may be incorrect. In MAP with fixed priori information, all pixels of the estimated image are related to the only fixed statistical parameters. Thus, the estimation results may seem to be bright or dark through the entire estimated image. In addition, both estimation techniques can reconstruct the only grayscale 3D images.

To solve these estimation problems, in this Letter, we propose a novel 3D color microscopy for microorganisms under photon-starved conditions using photon counting integral imaging and Bayesian estimation with adaptive priori information. We obtain the adaptive priori information by reconstructing 3D sliced images with various reconstruction depth planes. Then, applying MAP to each basic color channel, we estimate more accurate 3D color microscopy images.

A photon counting detector can be modeled statistically by a Poisson distribution since photons occur rarely in unit time and space^[5]. Figure 1 illustrates a mathematical model of a photon counting imaging system.

Sign up for Chinese Optics Letters TOC. Get the latest issue of Chinese Optics Letters delivered right to you！Sign up now

Figure 1.Mathematical model of our photon counting imaging system.

For computational simplicity, we use one-dimensional notation only. To extract photons from 3D scenes in a photon counting model, the original scene, $I (x)$ , is normalized because the unit energy of the scene and the controllable expected number of photons $(N_{p})$ are used to generate photons^[3,4]. Photon generation in our statistical model is described by the following $λ (x) = \frac{I (x)}{\sum_{x = 1}^{N_{x}} I (x)}$ (1) $C (x) | λ (x) \sim Poisson [N_{p} λ (x)],$ (2)where $λ (x)$ is the normalized irradiance of the detected scene, $N_{x}$ is the number of pixels in the image, and $C (x)$ is the number of photons extracted from the normalized scene. In this Letter, we refer to $C (x)$ as the photon-limited image.

In general, digital image has three basic color channels; red (R), green (G), and blue (B). Since each color channel has different mean optical frequencies and photons with different wavelengths carry different energy as per Eqs. (3) and (4), the photon generation process for each color channel may be different from each other^[5] ${\bar{N}}_{c} = γ_{c} W = \frac{η W}{h {\bar{ν}}_{c}},$ (3) $γ_{c} = \frac{η}{h {\bar{ν}}_{c}},$ (4)where ${\bar{N}}_{c}$ is the number of photons, $W$ is the energy incident on the photo-surface during measurement, h is Planck’s constant, ${\bar{ν}}_{c}$ is the mean optical frequency of the radiation, and $η$ is the quantum efficiency which represents the average number of photoevents produced by each incident photon $(η \geq 1)$ .

For 3D visualization of microorganisms under photon-starved conditions, we introduce an integral imaging technique. Figure 2 shows the pickup stage of integral imaging using synthetic aperture integral imaging (SAII)^[12] and an objective lens to capture the elemental images with high resolution.

Figure 2.Pickup stage of photon counting microscopy using SAII.

In SAII, each camera can detect the image with slightly different perspectives. Under photon-starved conditions, using SAII and Eqs. (1) and (2), multiple photon-limited images can be generated. Since the statistical distribution of each photon-limited image follows a Poisson distribution by a mathematical photon counting imaging model; the likelihood function can be constructed as per the following^[3,4] $L (λ) = \prod_{k = 1}^{K} \frac{{(N_{p} λ_{k})}^{C_{k}} e^{- N_{p} λ_{k}}}{C_{k}!},$ (5)where $λ = (λ_{1}, \dots, λ_{K})$ . Using MLE we can estimate each $λ_{k}$ as $C_{k} / N_{p}$ . Therefore, we can reconstruct 3D images under photon-starved conditions using estimated images and a computational volumetric reconstruction algorithm of integral imaging^[13] as per the following $I (x, z_{r}) = \frac{1}{N_{p} O (x)} \sum_{k = 1}^{K} C_{k} (x + Δ x (k - 1)),$ (6) $Δ x = \frac{N_{x} p f}{c_{x} z_{r}},$ (7)where $O (x)$ is the overlapping factor for reconstruction, $Δ x$ is the number of shifted pixels for superposition of a reconstruction process, $p$ is the gap between the cameras, $f$ is the focal length of the camera lens, $c_{x}$ is the image sensor size, and $z_{r}$ is the reconstruction depth.

However, since MLE uses no priori information (i.e., the priori information is assumed to be a uniform distribution), the estimated image is not accurate. To obtain a more accurate estimated image, Bayesian estimation such as MAP with fixed priori information^[4] was proposed. It uses a gamma distribution with $α$ and $β$ as the priori information because it is a conjugate priori of Poisson distribution. The priori information can be defined by the following^[4] $π (λ_{k}) = \frac{β^{α}}{Γ (α)} λ_{k}^{α - 1} e^{- β λ_{k}}, λ_{k} > 0 .$ (8)Multiplying Eq. (5) by Eq. (8), the posteriori distribution can be obtained. To estimate the image, the posteriori mean can be calculated by MAP as per the following ${\tilde{λ}}_{k} (x) = \frac{C_{k} (x) + α_{k}}{N_{p} (1 + β_{k})}, C_{k} (x) > 0,$ (9)where $α_{k} = μ_{k}^{2} / σ_{k}^{2}$ , $β_{k} = μ_{k} / σ_{k}^{2}$ , $μ_{k}$ is the mean of $λ_{k} (x)$ , and $σ_{k}^{2}$ is the variance of $λ_{k} (x)$ . Thus, using Eqs. (6) and (9), the reconstructed 3D images can be obtained as per the following $\tilde{I} (x, z_{r}) = \frac{1}{N_{p} O (x)} \sum_{k = 1}^{K} {\tilde{λ}}_{k} (x + Δ x (k - 1)) .$ (10)However, in MAP, because the only fixed statistical parameters of the priori distribution are used throughout the entire image, it may difficult to obtain the accurate 3D images at various reconstruction depths. To enhance the accuracy of the MAP estimation, we use adaptive priori information. Since the number of shifted pixels for overlapping multiple photon-limited images in reconstruction process, $Δ x$ , is varied corresponding to various reconstruction depths, $z_{r}$ , the statistical parameters can be adaptive at each reconstruction planes as per the following $μ_{z_{r}} (x) = \frac{1}{O (x)} \sum_{k = 1}^{K} λ_{k} (x + Δ x (k - 1)),$ (11) $σ_{z_{r}}^{2} (x) = E {[λ_{k} (x) - μ_{z_{r}} (x)]}^{2},$ (12) $α_{z_{r}} (x) = \frac{{[μ_{z_{r}} (x)]}^{2}}{σ_{z_{r}}^{2} (x)},$ (13) $β_{z_{r}} (x) = \frac{μ_{z_{r}} (x)}{σ_{z_{r}}^{2} (x)},$ (14)where $E [\cdot]$ is the expectation operator. From Eqs. (11)–(14), it is apparent that statistical parameters of the priori distribution, $α$ and $β$ , are functions of the pixel position, $x$ , and reconstruction depth, $z_{r}$ . Using these adaptive statistical parameters, the estimated images and reconstructed 3D images can be obtained as follows $λ_{k}^{'} (x) = \frac{C_{k} (x) + α_{z_{r}} (x)}{N_{p} (1 + β_{z_{r}} (x))}, C_{k} (x) > 0,$ (15) $I^{'} (x, z_{r}) = \frac{1}{N_{p} O (x)} \sum_{k = 1}^{K} λ_{k}^{'} (x + Δ x (k - 1)) .$ (16)To verify our proposed method, we carried out an optical experiment. A $7 \times 7$ camera array was used. The focal length of the camera lens was 105 mm. The moving gaps between cameras in the $x$ and $y$ directions were 2 and 1 mm, respectively. The microorganism was a ladybug. To magnify the microorganism, an objective lens ( $20 \times$ ) was used. Each recorded image had $2001 pixels \times 2001 pixels$ . The location of the magnified object image was 234 mm from the camera array.

To detect multiple images under photon-starved conditions, we normalized the recorded images by Eq. (1) and generate the color photon-limited images with $N_{p} = 20000$ by Eqs. (2) and (4) as shown in Fig. 3.

Figure 3.Recorded images: (a) original image; (b) photon-limited image with $N_{p} = 20000$ .

Then, using Eqs. (6), (10), and (16), the reconstructed 3D images using MLE, MAP with fixed statistical parameters, and MAP with adaptive statistical parameters can be obtained as shown in Fig. 4.

Figure 4.Reconstruction results by the following: (a) computational volumetric reconstruction; (b) MLE; (c) MAP with fixed statistical parameters; (d) MAP with adaptive statistical parameters.

As shown in Fig. 4(d), the reconstructed image by MAP with adaptive statistical parameters has better visual quality and is more accurate than the others. To evaluate the performance of our proposed method, we calculate the mean square error (MSE) for each method. For calculating the MSE, we use the reconstructed image using the original color images as shown in Fig. 4(a) as the reference image. Figure 5 shows the MSE results for each method. Therefore, it is noticed that our proposed method has better performance than the others.

Figure 5.MSE for various reconstruction depths.

In conclusion, we propose a novel 3D color microscopy for microorganisms under photon-starved conditions. Using priori information with adaptive statistical parameters and MAP, we obtain more accurate 3D images at various reconstruction depths. In addition, since photons with different color channels carry different energies to the image sensor, color photon counting integral imaging can be modeled considering this feature. Finally, we believe that our technique can be applied to various applications under photon-starved conditions.

References

[1] S. Yeom, B. Javidi, C.-W. Lee, E. Watson. Opt. Express, 15, 16189(2007).

[2] I. Moon, B. Javidi. Opt. Lett., 34, 731(2009).

[3] B. Tavakoli, B. Javidi, E. Watson. Opt. Express, 16, 4426(2008).

[4] J. Jung, M. Cho, D. K. Dey, B. Javidi. Opt. Lett., 35, 1825(2010).

[5] J. W. Goodman. Statistical Optics(1985).

[6] B. Javidi, F. Okano, J.-Y. Son. Three-Dimensional Imaging, Visualization, and Display Technology(2008).

[7] G. Lippmann. C. R. Acad. Sci., 146, 446(1908).

[8] B. Javidi, I. Moon, S. Yeom. Opt. Express, 14, 12096(2006).

[9] J.-S. Jang, B. Javidi. Opt. Lett., 29, 1230(2004).

[10] M. Cho, M. Daneshpanah, I. Moon, B. Javidi. Proc. IEEE, 99, 556(2011).

[11] H. Navarro, A. Dorado, G. Saavedra, M. M. Corral. J. Inf. Commun. Convergence Eng., 12, 89(2014).

[12] J.-S. Jang, B. Javidi. Opt. Lett., 27, 1144(2002).

[13] S.-H. Hong, J.-S. Jang, B. Javidi. Opt. Express, 12, 483(2004).