The quanta image sensor (QIS)^[1] is a modified CMOS image sensor (CIS)^[2] that contains a large number of photon-counting pixels, and oversamples both spatially and temporally. The specialized pixels in QISs, which have sub-diffraction-limit (SDL) size, are referred to as “jots.” Based on the bit depth of analog-to-digital converters (ADCs), QISs are categorized as single-bit QISs and multi-bit QISs.

To intuitively comprehend the concept of the QIS, a conceptual illustration is shown in Fig. 1. As shown in Fig. 1, the output of the QIS is a large spatial-temporal data cube that can be flexibly combined in subsequent image processing.

The main development direction of image sensors is high integration, high performance, and low cost. Therefore, the size of active pixels in CISs keeps shrinking to achieve higher resolution on a smaller chip size, which meets the needs of high integration and low cost. Nevertheless, the shrinkage of pixels causes the problem of full well capacity (FWC) decrease that leads to signal-to-noise ratio (SNR) and dynamic range (DR) decreases^[3], which limits the further development of CISs to high performance.

To alleviate the problem of FWC decrease, the QIS, which is a candidate for the next-generation solid image sensor, was first introduced in 2005^[4] and was then presented in 2011^[5]. Progress has been made in theoretical research and engineering realization of QISs in recent years. The required conditions to achieve photon and photoelectron counting based on active pixel image sensors were analyzed theoretically in Refs. [6, 7]. A theoretical model of signal, noise, photoresponse, SNR, and DR for single-bit and multi-bit QISs was established in Refs. [8, 9]. The bit error rate (BER) as a function of the exposure level and read noise for QISs was analyzed in Ref. [10], and the recommended input-referred read noise was lower than 0.15 e- r.m.s.. A modified 1/f noise model of the source follower in jots, which was fitted to the measured data, was established in Ref. [11]. A pump-gate jot with high conversion gain (CG) of 423 μV/e- and low read noise of 0.22 e- r.m.s. was presented in Refs. [12, 13]. A low-power readout circuit with a modified charge-transfer amplifier (CTA) was proposed in Ref. [14]. Moreover, a 1 Mjot 1040 fps QIS chip in the stacked backside-illuminated (BSI) process was reported in Refs. [15, 16].

The prospective application fields of QISs include consumer imaging, low-light imaging, security monitoring imaging, scientific imaging, medical imaging, and space imaging. When the image performance strongly depends on the absolute measurements of incident photons, especially in scientific and medical imaging, high accuracy photon and photoelectron counting is required. Therefore, the high accuracy photon and photoelectron counting characteristics of QISs should be studied further.

The photoresponse characteristic of QISs, which is referred as D–logH response, was theoretically derived in Ref. [9] and experimentally proven in Ref. [17]. On the one hand, for high dynamic range (HDR) applications, the overexposure latitude range of the D–logH response curve can be used to extend the DR of the QIS. On the other hand, for applications that need high accuracy photon and photoelectron counting, the non-linear response of overexposure latitude range will increase the counting error of the QIS in high light level, and the non-ideal characteristics, such as dark current and read noise, will increase the counting error of the QIS^[10] in low light level.

Therefore, the photoresponse and counting error characteristics of QISs, and the effects of non-linear response and non-ideal characteristics to them are needed to be investigated. The high accuracy photon and photoelectron counting exposure ranges of QISs are required to be determined theoretically for subsequent on-chip or off-chip calibrations, and the integration time optimization under different light level and jot size is one of the common methods.

In this paper, an integration time optimization method towards high accuracy photon and photoelectron counting is presented to guide the design and operation of single-bit and multi-bit QISs. The remainder of this paper is organized as follows. A signal chain model of single-bit and multi-bit QISs is established in Section 2. The photoresponse characteristics for ideal and realistic QISs are investigated in Section 3. The signal error rates towards photon and photoelectron counting are investigated in Section 4. An optimization method of integration time that can guide the design and operation of QISs is presented in Section 5. Finally, the conclusions are given in Section 6.

The signal chain model of single-bit and multi-bit QISs is illustrated in Fig. 2, which contains signal conversion processes of photons to electrons, electrons to voltages, and voltages to digital numbers. The parameters of the signal chain model in Fig. 2 are listed in Table 1.

As shown in Fig. 2 and Table 1, the input and output signals of the proposed model are k_ph and DN, respectively. In the jot, k_ph incident photons arrive at the pinned photodiode and generate k_phe photoelectrons on the condition that the quantum efficiency of the jot is QE. Then, k_phe photoelectrons and k_d dark signal electrons, which are generated from dark current, constitute together k_e total signal electrons. Then, k_e total signal electrons are transferred to the floating diffusion and converted to V_CG on the condition that the conversion gain of the jot is CG. In the readout circuit, V_CG is converted to V_RO on the condition that the read noise of the readout circuit is v_n. In the ADC, V_RO is converted to DN on the condition that the quantizer threshold of the ADC is v_th.

It is noted that the electron-referred values (i.e., the normalized values) of V_CG, V_RO, v_n, and v_th are U_CG, U_RO, u_n, and u_th, respectively. In most cases, the normalized values are introduced because it is more convenient to use them in mathematical derivation. The unit of ro (i.e., ADU) is equivalent to p or e-, when the ADC output digital numbers are compared with photons or photoelectrons. To analyze the proposed signal chain model mathematically, a series of probability distributions are introduced in the following sections.

In the proposed signal chain model, the total signal electrons consist of photoelectrons and dark signal electrons, which are generated from incident photons and dark current in the jot, respectively. As described in Refs. [13, 18], benefiting from the jot device structure and the QIS operating characteristic, the image lag and crosstalk of pump-gate jots in the QIS are acceptable for photon counting and photoelectron counting applications under different exposure levels. Therefore, other sources of total signal electrons, including image lag, optical crosstalk, and electronic crosstalk, are not considered in the proposed model.

The emission of photons from most light sources is well-described by Poisson distribution^[9]. Thus, over the integration time τ, the probability mass function (PMF) that k_ph incident photons arrive at the jot is defined as:

$ {{p}}\left[{k}_{\rm{ph}}\right]=\frac{{{\mu }_{\rm{ph}}}^{{k}_{\rm{ph}}}}{{k}_{\rm{ph}}!}{\rm{e}}^{-{\mu }_{\rm{ph}}}, $  (1)

where μ_ph is the mean value of incident photons, and k_ph is a nonnegative integer.

The quantum efficiency of the jot QE is defined as the number of photoelectrons divided by the number of incident photons. The process that photons convert to photoelectrons obeys binomial distribution^[7]. Thus, the PMF that k_phe photoelectrons generate from k_ph incident photons under QE is defined as:

$ {p}\left[{k}_{\rm{phe}};{k}_{\rm{ph}},{\rm{QE}}\right]=\frac{{k}_{\rm{ph}}!}{{k}_{\rm{phe}}!\left({k}_{\rm{ph}}-{k}_{\rm{phe}}\right)!}{\rm{QE}}^{{k}_{\rm{phe}}}(1-{{\rm{QE}})}^{{k}_{\rm{ph}}-{k}_{\rm{phe}}}, $  (2)

where k_phe and k_ph are nonnegative integers, and k_phe is less than or equal to k_ph.

Then, the PMF that k_phe photoelectrons are generated in the jot under μ_ph and QE, which is the convolution of two mutually independent PMFs Eqs. (1) and (2), is derived as:

$\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! {p}\left[{k}_{\rm{phe}}\right]=\sum\limits_{{k}_{\rm{ph}}={k}_{\rm{phe}}}^{\infty }{p}\left[{k}_{\rm{ph}}\right]{p}\left[{k}_{\rm{phe}};{k}_{\rm{ph}},\rm{QE}\right] $  (3)

$ =\frac{{(\rm{QE}\cdot {\mu }_{\rm{ph}})}^{{k}_{\rm{phe}}}}{{k}_{\rm{phe}}!}{\rm{e}}^{-{\mu }_{\rm{ph}}}\sum\limits_{{k}_{\rm{ph}}={k}_{\rm{phe}}}^{\infty }\frac{1}{({k}_{\rm{ph}} \!-\!{k}_{\rm{phe}})!}{\left[(1 \! - \! \rm{QE}){\mu }_{\rm{ph}}\right]}^{{k}_{\rm{ph}}-{k}_{\rm{phe}}} $  (4)

$ =\frac{({\rm{QE}\cdot {\mu }_{\rm{ph}})}^{{k}_{\rm{phe}}}}{{k}_{\rm{phe}}!}{\rm{e}}^{-{\mu }_{\rm{ph}}}{\rm{e}}^{(1-\rm{QE}){\mu }_{\rm{ph}}} $  (5)

$ \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! =\frac{({\rm{QE}\cdot {\mu }_{\rm{ph}})}^{{k}_{\rm{phe}}}}{{k}_{\rm{phe}}!}{\rm{e}}^{-\rm{QE}\cdot {\mu }_{\rm{ph}}} $  (6)

$\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! =\frac{{{\mu }_{\rm{phe}}}^{{k}_{\rm{phe}}}}{{k}_{\rm{phe}}!}{\rm{e}}^{-{\mu }_{\rm{phe}}}, $  (7)

where obeys Poisson distribution, and μ_phe is the mean value of the photoelectrons.

Similarly, the emission of dark signal electrons that generate from dark current in active pixels also obeys Poisson distribution. Thus, over the integration time τ, the PMF that k_d dark signal electrons are generated in the jot is defined as:

$ {p}\left[{k}_{\rm{d}}\right]=\frac{{{\mu }_{\rm{d}}}^{{k}_{\rm{d}}}}{{k}_{\rm{d}}!}{\rm{e}}^{-{\mu }_{\rm{d}}}, $  (8)

where μ_d is the mean value of dark signal electrons, and k_d is a nonnegative integer.

Then, the PMF that k_e total signal electrons are generated in the jot, which is the convolution of two mutually independent PMFs Eqs. (7) and (8), is derived as:

$\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! {p}\left[{k}_{\rm{e}}\right]=\sum\limits_{{k}_{\rm{phe}}=0}^{{k}_{\rm{e}}}{p}\left[{k}_{\rm{phe}}\right]{p}\left[{k}_{\rm{d}}={k}_{\rm{e}}-{k}_{\rm{phe}}\right] $  (9)

$ =\frac{1}{{k}_{\rm{e}}!}{\rm{e}}^{-({\mu }_{\rm{phe}}+{\mu }_{\rm{d}})}\sum\limits_{{k}_{\rm{phe}}=0}^{{k}_{\rm{e}}}\frac{{k}_{\rm{e}}!}{{k}_{\rm{phe}}!\left({k}_{\rm{e}}-{k}_{\rm{phe}}\right)!}{{\mu }_{\rm{phe}}}^{{k}_{\rm{phe}}}{{\mu }_{\rm{d}}}^{{k}_{\rm{e}}-{k}_{\rm{phe}}} $  (10)

$ =\frac{{({\mu }_{\rm{phe}}+{\mu }_{\rm{d}})}^{{k}_{\rm{e}}}}{{k}_{\rm{e}}!}{\rm{e}}^{-({\mu }_{\rm{phe}}+{\mu }_{\rm{d}})} $  (11)

$\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! =\frac{{{\mu }_{\rm{e}}}^{{k}_{\rm{e}}}}{{k}_{\rm{e}}!}{\rm{e}}^{-{\mu }_{\rm{e}}}, $  (12)

where also obeys Poisson distribution, μ_e is the mean value of total signal electrons, and k_e is a nonnegative integer.

The total signal electrons are transferred from pinned photodiode (PPD) to floating diffusion (FD) and converted to V_CG in the jot, when the conversion gain of the jot is CG that is calculated as:

$ {\rm{CG}}=\frac{q}{{C}_{{\rm{FD}}}}, $  (13)

where q is the electron charge, C_FD is the total capacitance of FD, and the unit of CG is usually μV/e-.

Then, the voltage-referred jot output is defined as:

$ {V}_{{\rm{CG}}}={\rm{CG}}\cdot {k}_{\rm{e}}. $  (14)

It is convenient to use normalized values to simplify the analysis of the proposed model. Thus, the normalized value (i.e., the electron-referred value) of V_CG is calculated as^[10]:

$ {U}_{{\rm{CG}}}=\frac{{V}_{{\rm{CG}}}}{\rm{CG}}. $  (15)

In the subsequent derivation, a similar method is used to convert the voltage-referred values V_RO, v_n, and v_th to the normalized values U_RO, u_n, and u_th, respectively.

The readout circuit in the proposed model contains the source follower (SF), the amplifying circuit, and the correlation double sampling (CDS) circuit before the ADC. The voltage-referred read noise of the readout circuit v_n, which contains thermal noise and 1/f noise, is normalized as^[10]:

$ {u}_{\rm{n}}=\frac{{v}_{\rm{n}}}{{\rm{CG}}}. $  (16)

It is noted that only temporal noise is considered in the proposed model. In other words, the spatial noise (i.e., fixed pattern noise (FPN), including dark signal nonuniformity (DSNU) and photoresponse nonuniformity (PRNU)), which may cause by QE, CG, u_n, and u_th nonuniformities among jots array, are not considered in the proposed model.

The main approach of the QIS to achieve photon counting and photoelectron counting is to increase the conversion gain of jots to reduce the electron-referred read noise lower than 0.5 e- r.m.s. (i.e., deep sup-electron read noise level). For example, when v_n is equal to 100 μV r.m.s., CG is required to be higher than 200 μV/e-, i.e., C_FD is required to be lower than 0.8 fF, to make sure u_n is lower than 0.5 e- r.m.s..

When the read noise u_n obeys Gaussian distribution, the component probability density function (PDF) that the normalized readout circuit output U_RO generates from the normalized jot output U_CG is defined as^[10]:

$ P\left[{U}_{{\rm{RO}}};{U}_{{\rm{CG}}}\right]=\frac{1}{\sqrt{2\pi {{u}_{\rm{n}}}^{2}}}\exp\left[-\frac{{({U}_{{\rm{RO}}}-{U}_{{\rm{CG}}})}^{2}}{2{{u}_{\rm{n}}}^{2}}\right]. $  (17)

Using Eqs. (12), (14), (15), and (17), the PDF that the normalized readout circuit output U_RO is generated, when the mean value of total signal electrons is μ_e, is derived as:

$ P\left[{U}_{{\rm{RO}}}\right]=\sum\limits_{{k}_{\rm{e}}=0}^{\infty }\frac{{p}\left[{k}_{\rm{e}}\right]}{\sqrt{2\pi {{u}_{\rm{n}}}^{2}}}\exp\left[-\frac{{({U}_{{\rm{RO}}}-{k}_{\rm{e}})}^{2}}{2{{u}_{\rm{n}}}^{2}}\right], $  (18)

where U_RO is a real number.

The PDF P[U_RO] as a function of U_RO on the condition that μ_e = 2 e- and u_n = 0.2 e- r.m.s. is shown in Fig. 3 as an example.

In the process of quantization, the normalized readout circuit output U_RO is converted to the ADC output DN based on different quantization levels. For an n-bit QIS, there are 2ⁿ quantization levels and 2ⁿ – 1 quantization boundaries. The effective full well capacity of the QIS is generally limited by the bit depth of ADC and equal to 2ⁿ– 1.

As shown in Fig. 3, for a 2-bit QIS, four quantization levels are N₀, N₁, N₂, and N₃, which correspond to 0, 1, 2, and 3 ADU, respectively. When the normalized quantizer threshold u_th = 0.5 e-, three normalized quantization boundaries are 0.5, 1.5, and 2.5 e-. When U_RO∈(–∞, 0.5 e-], DN = 0 ADU; when U_RO∈(0.5 e-, 1.5 e-], DN = 1 ADU; when U_RO∈(1.5 e-, 2.5 e-], DN = 2 ADU; when U_RO∈(2.5 e-, +∞], DN = 3 ADU.

Thus, for an n-bit QIS, the PMF that the ADC output digital numbers DN is generated, when the normalized quantizer threshold is u_th, is defined as:

$ {p}\left[ {{\rm{DN}}} \right] = \left\{ \begin{array}{l} \int_{ - \infty }^{{U_{{\rm{th}}1}}} P\left[ {{U_{{\rm{RO}}}}} \right]{\rm{d}}{U_{{\rm{RO}}}},\;\;\;\;\;\;\;{\rm{DN}} = {\rm{}}0,\\ \int_{{U_{{\rm{th}}2}}}^{{U_{{\rm{th}}1}}} P\left[ {{U_{{\rm{RO}}}}} \right]{\rm{d}}{U_{{\rm{RO}}}},\;\;\;\;\;\;\;0 \lt , {\rm{DN}} \lt , {\rm{FW}}{{\rm{C}}_{{\rm{eff}}}},\\ \int_{{U_{{\rm{th}}2}}}^{ + \infty } P\left[ {{U_{{\rm{RO}}}}} \right]{\rm{d}}{U_{{\rm{RO}}}},\;\;\;\;\;\;\;{\rm{DN}} = {\rm{FW}}{{\rm{C}}_{{\rm{eff}}}},\\ 0,\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\,\quad{\rm{DN}} \gt; {\rm{FW}}{{\rm{C}}_{{\rm{eff}}}}, \end{array} \right. $  (19)

where the higher quantization boundary U_th1 = DN + u_th, the lower quantization boundary U_th2 = DN – (1 – u_th), the effective full well capacity FWC_eff = 2ⁿ– 1, and DN is a nonnegative integer.

Conventional CISs generally have linear response, and some CISs can achieve logarithmic and linear-logarithmic response through pixel or circuit structure design. In contrast from these CISs, the QIS has a special photoresponse characteristic, which is referred to as D–logH response. For high dynamic range applications, the overexposure latitude range of the D–logH response curve is used to extend the dynamic range of the QIS^[9]. For applications that need high accuracy photon counting and photoelectron counting, the high accuracy linear range of the D–logH response curve is required to be determined in theory.

As shown in Fig. 1, by reconstructing a specified spatial-temporal data cube of jots into a single data point of a pixel in the output image, the trade-off between the equivalent full well capacity, spatial resolution, and temporal resolution can be flexibly achieved. The spatial and temporal oversampling coefficients of the QIS are defined as M_S and M_T, respectively, and the total oversampling coefficient of the QIS is calculated as:

$ M={M}_{\rm{S}}{M}_{\rm{T}}, $  (20)

where M has various spatial-temporal combinations. For example, when M = 512, a 4 × 4 × 32, 8 × 8 × 8, 16 × 16 × 2, et al. sub-cube of jots is alternative.

Assuming the jots used for reconstruction are completely equivalent (i.e., no DSNU and PRNU among jots array), the signal value of ADC output is calculated as:

$ {S}_{{\rm{DN}}}=\sum\limits_{i=0}^{M}{S}_{{\rm{DN}},i}=M\cdot \sum\limits_{\rm{DN}=0}^{\infty }\rm{DN}\cdot {p}\left[\rm{DN}\right], $  (21)

where is given by Eq. (19).

Similarly, the saturated signal value of ADC output is calculated as:

$ {S_{{\rm{DN}},{\rm{sat}}}} = {\rm{FW}}{{\rm{C}}_{{\rm{equ}}}} = M \cdot {\rm{FW}}{{\rm{C}}_{{\rm{eff}}}} = M\left( {{2^n} - 1} \right), $  (22)

where FWC_equ and FWC_eff is the equivalent and effective full well capacity respectively, and n is the bit depth of QISs (i.e., the bit depth of ADCs in those QISs). For example, for a 3-bit QIS, when M_S = 4 × 4, M_T = 8 and M = 4 × 4 × 8 = 128, FWC_equ = M · FWC_eff = 896 e-.

In Ref. [9], in the D-logH function, H denotes quanta exposure, which is defined as the average number of photons or photoelectrons arrive at a jot over the integration period; and D denotes bit density, which is defined as the number of saturated jots divided by the number of total jots for 1-bit QISs.

In this paper, the D–logH function is extended to n-bit QISs. H denotes quanta exposure, which is defined as the mean value of photons that arrive at a jot over the integration time τ. Thus, H is equal to μ_ph. When QE = 1 e-/p, H is also equal to μ_phe × 1 p/e-. Meanwhile, D denotes signal density, which is defined as:

$ D=\frac{{S}_{{\rm{DN}}}}{{S}_{{\rm{DN}},{\rm{sat}}}}=\frac{\displaystyle\sum\limits_{\rm{DN}=0}^{\infty }\rm{DN}\cdot {p}\left[\rm{DN}\right]}{{2}^{n}-1}, $  (23)

where S_DN and S_DN,sat are given by Eqs. (21) and (22) respectively.

The ideal D-logH response curves for 1-bit, 2-bit, 3-bit, 4-bit, and 5-bit QISs are shown in solid lines in Fig. 4, where QE = 1 e-/p, μ_d = 0 e-, u_n = 0 e- r.m.s., and u_th = 0.5 e-. The ideal linear response curves for 1-bit to 5-bit CISs are shown in dashed lines in Fig. 4, where the output signal is equal to the input signal (i.e., the ideal accuracy photon counting is achieved).

As shown in Fig. 4, when D = 0.99, the saturated quanta exposure H_sat of 1-bit to 5-bit QISs are equal to 4.7, 7.3, 13, 22, and 39 p, respectively. Meanwhile, H_sat of 1-bit to 5-bit CISs are equal to 0.99, 2.97, 6.93, 14.85, and 30.69 p, respectively. Thus, compared with the n-bit CIS, the n-bit QIS has a higher H_sat and a greater potential in HDR applications in theory.

As H increases from 0.01 to 100 p, the QIS gradually enters in the nonlinear response range (the non-overlap range of solid line and dashed line) from the linear response range (the overlap range of solid line and dashed line). As n increases from 1 to 5, the linear and nonlinear response ranges of 1-bit to 5-bit QISs are extended and narrowed respectively. Thus, QISs with higher bit depth are inferred to have a better performance in applications that need high accuracy photon counting and photoelectron counting in theory.

For realistic QISs, QE < 1 e-/p, μ_d> 0 e-,u_n > 0 e- r.m.s., and u_th = 0.5 e-. In the realistic response curve for an n-bit QIS, H is equal to μ_ph, and D is calculated as Eq. (23). The realistic response curves for the 3-bit QIS under five different conditions, which are listed in Table 2, are shown in Fig. 5.

Based on Fig. 5, the effects of QE, μ_d, and u_n to realistic response curves for the QIS are qualitatively analyzed, as follows:

(1) Curve 1 (ideal) is the ideal response curve for the 3-bit QIS as a comparison;

(2) Curve 2 (QE = 0.8 e-/p) can be approximatively considered to be obtained by shifting curve 1 to the right. This means that when QE < 1 e-/p, a higher H is required to get the same D in curve 1;

(3) In curve 3 (μ_d = 0.01 e-), as H increases from 0.01 p to 100 p, curve 3 gradually coincides with curve 1, the difference of D between curve 3 and curve 1 gradually decreases, i.e., μ_d > 0 e- has an obvious effect on low light level ( H < 0.1 p);

(4) In curve 4 (u_n = 0.3 e- r.m.s.), u_n > 0 e- r.m.s. has a similar effect on low light level ( H < 0.1 p) compared with μ_d > 0 e- in curve 3;

(5) Curve 5 (QE = 0.8 e-/p, μ_d = 0.01 e-, and u_n = 0.3 e- r.m.s.) combines the effects of QE < 1 e-/p, μ_d > 0 e-, and u_n > 0 e- r.m.s.. When H is lower than 0.25 p, D in curve 5 is higher than curve 1, μ_d > 0 e- and u_n > 0 e- r.m.s. are dominant. When H is higher than 0.25 p, D in curve 5 is lower than curve 1, QE < 1 e-/p is dominant.

In summary, the lower boundary of the linear response range of the QIS is limited by μ_d and u_n, the higher boundary of the above range is limited by the bit depth of QISs n, and this range shifts to the right when QE < 1 e-/p. To further study the high accuracy linear ranges towards photon counting and photoelectron counting of ideal and realistic response curves, and the effects of QE, μ_d, and u_n to the above ranges of realistic response curves, a quantitative analysis is performed in Section 4.

In this paper, the signal error rate is defined as the absolute value of the difference between the input and output signals divided by the input signal, which is used as the qualifying factor to evaluate the accuracy of photon counting or photoelectron counting for the QIS.

Assuming that the jots used for reconstruction are completely equivalent, like Eq. (21), the signal value of incident photons is calculated as:

$ {S}_{\rm{ph}}=M\sum\limits_{{k}_{\rm{ph}}=0}^{\infty }{k}_{\rm{ph}}\cdot {p}\left[{k}_{\rm{ph}}\right]=M{\mu }_{\rm{ph}}. $  (24)

Similarly, the signal value of photoelectrons is calculated as:

$ {S}_{\rm{phe}}=M\sum\limits_{{k}_{\rm{phe}}=0}^{\infty }{k}_{\rm{phe}}\cdot {p}\left[{k}_{\rm{phe}}\right]={M\mu }_{\rm{phe}}. $  (25)

And the signal error rate between S_DN and S_ph is defined as:

$ {\rm{SER}}_{\rm{ph}}=\frac{\left|{S}_{{\rm{DN}}}-{S}_{\rm{ph}}\right|}{{S}_{\rm{ph}}}=\frac{\left|\sum\limits_{\rm{DN}=0}^{\infty }{\rm{DN}}\cdot {p}\left[{\rm{DN}}\right]-{\mu }_{\rm{ph}}\right|}{{\mu }_{\rm{ph}}}, $  (26)

where SER_ph is the qualifying factor of evaluating the accuracy of photon counting in the QIS. When SER_ph is decreased, the difference between S_DN and S_ph is decreased, and the accuracy of photon counting is increased.

The signal error rate between S_DN and S_phe is defined as:

$ {\rm{SER}}_{\rm{phe}}=\frac{\left|{S}_{{\rm{DN}}}-{S}_{\rm{phe}}\right|}{{S}_{\rm{phe}}}=\frac{\left|\sum\limits_{\rm{DN}=0}^{\infty }{\rm{DN}}\cdot {p}\left[{\rm{DN}}\right]-{\mu }_{\rm{phe}}\right|}{{\mu }_{\rm{phe}}}, $  (27)

where SER_phe is the qualifying factor of evaluating the accuracy of photoelectron counting in the QIS. When SER_phe is decreased, the difference between S_DN and S_phe is decreased, and the accuracy of photoelectron counting is increased.

In this paper, based on Eqs. (26) and (27), the μ_ph value interval corresponding to SER_ph < 0.01 is defined as the high accuracy photon counting exposure range (hereinafter to be referred as R_ph) or high accuracy linear range of response curves, and the μ_phe value interval corresponding to SER_phe < 0.01 is defined as the high accuracy photoelectron counting exposure range (hereinafter to be referred as R_phe).

As for n-bit ideal QISs, QE = 1 e-/p, μ_d = 0 e-, u_n = 0 e- r.m.s., and u_th = 0.5 e-. In this case, μ_phe = μ_ph × 1 e-/p, and SER_phe = SER_ph, i.e., photon counting is equivalent to photoelectron counting when QE = 1 e-/p. And SER_ph as a function of μ_ph for ideal 1-bit to 5-bit QISs is shown in Fig. 6.

As shown in Fig. 6, for a given n, when μ_ph is increased, SER_ph is increased; for a given μ_ph, when n is increased, SER_ph is decreased; for ideal 1-bit to 5-bit QISs, R_ph is (–∞, 0.021 p), (–∞, 0.72 p), (–∞, 3.4 p), (–∞, 10 p), and (–∞, 25 p), respectively; in other words, as n increases from 1 to 5, R_ph is extended from (–∞, 0.021 p) to (–∞, 25 p).

Based on these results, it can be proven theoretically that increasing the bit depth of the QIS is an effective approach to improve the performance of QISs in applications that need high accuracy photon counting and photoelectron counting.

As for n-bit realistic QISs, QE < 1 e-/p, μ_d > 0 e-, u_n > 0 e- r.m.s., and u_th = 0.5 e-. In this case, photon counting is inequivalent to photoelectron counting. To determine the R_ph of realistic QISs, SER_ph as a function of μ_ph for the 3-bit QIS under five different conditions, which are listed in Table 2, is shown in Fig. 7.

Based on Fig. 7, the R_ph of curve 1 to 5 is (–∞, 3.4 p), N/A, (0.99 p, 3.6 p), (1.2 p, 3.4 p), and (0.23p, 0.26p), respectively. And the effects of QE, μ_d, and u_n to the R_ph of the realistic QIS are quantitative analyzed as follows:

(1) Curve 1 (ideal) is the ideal SER_ph curve for the 3-bit QIS as a comparison;

(2) In curve 2 (QE = 0.8 e-/p), when μ_ph ≤ 1.6 p, SER_ph remains constant at 0.2; when μ_ph > 1.6 p, SER _ph is gradually increased to 1. This means that QE has significant effect on SER_ph, and QE is the main limit factor of high accuracy photon counting for the QIS;

(3) In curve 3 (μ_d = 0.01 e-), as μ_ph increases from 0.01 p to 100 p, SER_ph first decreases from a high value to the minimum value and then gradually increases to 1. The minimum value point (i.e., the optimal point for photon counting) is (μ_ph = 2.7 p, SER_ph = 2.23 × 10^–4). Compared with curve 1, μ_d > 0 e- mainly increases the SER _ph on low light level (H < 0.1 p);

(4) In curve 4 (u_n = 0.3 e- r.m.s.), u_n > 0 e- r.m.s. has a similar effect on SER _ph compared with μ_d> 0 e- in curve 3. The minimum value point of curve 4 is (μ_ph = 2.4 p, SER_ph = 2.14 × 10^–4);

(5) In curve 5 (QE = 0.8, μ_d = 0.01 e-, and u_n = 0.3 e- r.m.s.), the effects of QE < 1 e-/p, μ_d > 0 e-, and u_n > 0 e- r.m.s. are combined. As μ_ph increases from 0.01 p to 100 p, SER_ph first decreases and then increases. The minimum value point of curve 5 is (μ_ph = 0.24 p, SER_ph = 4.37 × 10^–3). When μ_ph < 0.13 p, SER _ph > 0.2, the effects of μ_d = 0.01 e- and u_n = 0.3 e- r.m.s. are dominant; when 0.13 p ≤ μ_ph ≤ 3.5 p, 4.37 × 10^–3 ≤ SER_ph ≤ 0.2, the effect of QE = 0.8 e-/p is dominant; when μ_ph > 3.5 p, SER _ph is gradually increased to 1 due to the saturation of jots.

The cause of optimal point in curve 5 can be explained through the following example: assuming that 10 photons arrive at the jot and convert to 9 photoelectrons under QE; if 1 dark signal electron is generated, then the number of total signal electrons is 10; thus, a dark signal electron may fill the electron loss due to QE and cause SER_ph decrease in some special cases.

In summary, when QE is low, R_ph is limited by QE; when QE is high enough, such as 0.95 e-/p^[7], the lower boundary of R_ph is limited by μ_d and u_n, and the higher boundary of R_ph is limited by the saturated quanta exposure (i.e., the bit depth of QISs). Therefore, the main approach of extending the R_ph of realistic QISs is increasing QE and n, and decreasing μ_d and u_n.

To determine the R_phe of realistic QISs, SER_phe as a function of μ_ph for the 3-bit QIS under five different conditions, which are listed in Table 2, is shown in Fig. 8.

As shown in Fig. 8, the R_phe of curve 1 to 5 is (–∞, 3.4 p), (–∞, 4.3 p), (0.99 p, 3.6 p), (1.2 p, 3.4 p), and (2.1 p, 4.5 p), respectively. It is noted that since QE = 1 e-/p and SER_phe = SER_ph, curve 1, 3, and 4 in Fig. 8 are the same as Fig. 7.

Curve 2 (QE = 0.8 e-/p) can be approximatively considered to be obtained by shifting curve 1 (ideal) to the right (i.e., R_phe shifts to the right when QE < 1 e-/p).

In curve 5 (QE = 0.8 e-/p, μ_d = 0.01 e-, and u_n = 0.3 e- r.m.s.), the effects of QE < 1 e-/p, μ_d > 0 e-, and u_n> 0 e- r.m.s. are combined. Asμ_ph increases from 0.01 to 100 p, SER_phe first decreases and then increases. The minimum value point of curve 5 is (μ_ph = 3.5 p, SER_phe = 1.45 × 10^–4). When μ_ph < 3.5 p, SER _phe > 1.45 × 10 ^–4, the effects of μ_d = 0.01 e- and u_n = 0.3 e- r.m.s. are dominant; when μ_ph ≥ 3.5 p, curve 5 gradually overlaps with curve 2, and SER_phe is finally increased to 1 due to the saturation of jots.

In summary, the lower boundary of R_phe is limited by μ_d and u_n, the higher boundary of R_phe is limited by the bit depth of QISs, and R_phe shifts to the right when QE < 1 e-/p. Therefore, the main approach of extending the R_phe of realistic QISs is increasing n, and decreasing μ_d and u_n.

As discussed in Section 4, by calculating SER_ph and SER_phe, the μ_ph and μ_phe value interval corresponding to SER_ph < 0.01 and SER _phe < 0.01 (i.e., R_ph and R_phe) are determined.

In imaging applications, under the specified illuminance level and pixel size, by adjusting the integration time of the QIS, μ_ph and μ_phe can be adjusted to R_ph and R_phe, respectively, and the high accuracy photon counting and photoelectron counting can be achieved.

To propose the integration time optimization method, the illuminance of the light source I_lux should be converted to the mean value of incident photons μ_ph at first.

The conversion formula between I_lux and the mean value of photons that generate from the light source μ_s is defined as^[19]:

$ {\mu _{\rm{s}}}{\rm{}} = {\rm{}}\frac{{{I_{{\rm{lux}}}}{A_{{\rm{jot}}}}\tau }}{{K{E_{{\rm{ph}}}}}}{\rm{}} = {\rm{}}\frac{{{I_{{\rm{lux}}}}{A_{{\rm{jot}}}}\tau \lambda }}{{Khc}}, $  (28)

where E_ph denotes the energy of a single photon, A_jot denotes the area of a single jot, τ denotes the integration time of jots, K denotes the conversion coefficient, h = 6.626 × 10^–34 J∙s denotes Planck constant, c = 2.998 × 10⁸ m/s denotes the speed of light in vacuum, λ denotes the wavelength of photons.

The conversion formula between μ_s and μ_ph is defined as^[19]:

$ {\mu _{{\rm{ph}}}} = \frac{{T\left( {1 - R} \right) \cdot {\rm{FF}}}}{{4{F_{\rm{N}}}^2}}{\mu _{\rm{s}}}, $  (29)

where F_N denotes the F number of lens, T denotes the transmittance of lens, R denotes the reflectance of the image sensor surface, and FF denotes the fill factor of the jots.

When λ = 555 nm, K = 683 lm∙W^–1 = 683 lux∙m²∙W^–1[19], and let F_N = 2.8, T = 0.9, R = 0.1, and FF = 0.9. Using Eqs. (28) and (29), the conversion formula between I_lux and μ_ph is calculated as:

$ {\mu _{{\rm{ph}}}} = L{I_{{\rm{lux}}}}{A_{{\rm{jot}}}}\tau , $  (30)

where L is the corrected parameter of the specified QIS, and L = 9.51 × 10^–5 lux^–1∙μm^–2∙μs^–1 in this case.

For an ideal optical imaging system, the perfect lens focuses the point light source to a diffraction-limit spot, which is referred to as Airy disk, and the spot is surrounded by higher order diffraction rings. The diameter of Airy disk is calculated as^[4]:

$ {D}_{\rm{A}}=2.44\lambda {F}_{\rm{N}}. $  (31)

When λ = 555 nm and F_N = 2.8, D_A = 3.8 μm. To intuitively describe the geometric dimensioning of the Airy disk and jot array, and make a physical explanation of the relationship between I_lux, A_jot, and τ, a conceptual illustration is shown in Fig. 9.

As shown in Fig. 9, when A_jot = 1 μm² or A_jot = 0.25 μm², the same Airy disk covers 9 or 44 jots, respectively. This means that under specified I_lux and τ, when A_jot is larger, the jot has a higher probability of collecting more incident photons. In other words, when I_lux and A_jot are equal to specified values, μ_ph or μ_phe can be adjusted to be within R_ph or R_phe by optimizing integration time τ.

To further describe the integration time optimization method for QISs, the correlation parameters of the 1 Mjot QIS chip in Ref. [16] are listed in Table 3.

Based on the parameters listed in Table 3, QE = 0.79 e-/p, μ_d = 1.6 × 10^–6 e- to 4.8 × 10^–3 e- (assuming τ = 10 μs to 30 ms), u_n = 0.21 e- r.m.s., and u_th = 0.5 e-. SER_ph as a function of μ_ph is shown in Fig. 10, and SER_phe as a function of μ_ph is shown in Fig. 11. For solid lines, QE = 0.79 e-/p, μ_d = 1.6 × 10^–6 e-, u_n = 0.21 e- r.m.s., and u_th = 0.5 e-; for dashed lines, QE = 0.79 e-/p, μ_d = 4.8 × 10^–3 e-, u_n = 0.21 e- r.m.s., and u_th = 0.5 e-.

As shown in Fig. 10, for solid lines, R_ph is (0.035 p, 0.039 p) for 1-bit QIS, and R_ph is (0.038 p, 0.042 p) for 2-bit to 5-bit QISs. For dashed lines, R_ph is (0.052 p, 0.057 p) for 1-bit QIS, and R_ph is (0.059 p, 0.065 p) for 2-bit to 5-bit QISs.

As shown in Fig. 11, for solid lines, R_phe is (0.13 p, 0.17 p), (0.55 p, 1.1 p), (0.65 p, 4.3 p), (0.65 p, 13 p), and (0.65 p, 31 p) for 1 to 5 bit QISs, respectively. For dashed lines, R_phe is (0.17 p, 0.2 p), (0.74 p, 1.2 p), (1 p, 4.4 p), (1 p, 13 p), and (1 p, 31 p) for 1 to 5 bit QISs, respectively.

To sum up, as for the QISs in Figs. 10 and 11, QE is required to be further increased to extend R_ph; the dark current in jots is small enough that μ_d has a slight effect on R_phe even for a long integration time τ; u_n is required to be further decreased to extend R_phe; and a higher bit depth of QISs n is suggested to extend R_phe.

As listed in Table 3, the jot area A_jot = 1.21 μm². For a more advanced process, A_jot is predicted to be smaller in the future. The range of A_jot is assumed as 0.1 to 1.5 μm² in this paper. In addition, the acceptable integration time τ (hereinafter to be referred as R_τ) is assumed as 10 to 30 000 μs.

For the 3-bit QIS in Fig. 11, R_phe is (1 p, 4.4 p). Based on Eq. (30), the relationship between A_jot, μ_ph, and τ under different illuminance level I_lux is shown in Fig. 12.

As shown in Fig. 12, the area in orange denotes τ < 10 μs, and τ is below R_τ; the area in blue denotes 10 μs ≤ τ ≤ 30 000 μs, and τ is within the R_τ; the area in yellow denotes τ > 30 000 μs, and τ is above R_τ. As I_lux increases from 0.1 to 10 000 lux, the area in blue increases firstly and then decreases.

When I_lux is low, such as 0.1 lux, τ is required to reach a higher value that may be above R_τ; when I_lux is high, such as 10 000 lux, τ is required to reach a lower value that may be below R_τ. Thus, it will be more difficult to ensure that the QIS is working within R_phe, when the light level is too low or too high. Combined with the result in Section 4, when R_ph or R_phe is extended, the integration time optimization difficulty will be decreased. In addition, when I_lux is lower, a larger A_jot will decrease the optimization difficulty; when I_lux is higher, a smaller A_jot will decrease the optimization difficulty. For example, when I_lux = 1 lux and A_jot = 1.21 μm², the recommended value interval of τ is (8.8 ms, 30 ms), and when I_lux = 10 000 lux and A_jot = 0.21 μm², the recommended value interval of τ is (10 μs, 21.3 μs).

In summary, the trade-off between A_jot, μ_ph, and τ under different I_lux is needed to be considered, when the QIS is designed and operated. By integration time optimization, the QIS can work within R_ph or R_phe. The main approach of decreasing the optimization difficulty is to extend R_ph or R_phe, which required higher quantum efficiency, higher bit depth of QISs, lower dark current, and lower read noise. Therefore, when the correlation parameters of QISs are determined, the similar integration time optimization method can be performed to guide the design and operation of QISs.

In this paper, a signal chain model of single-bit and multi-bit QISs, which contains signal conversion processes of photons, electrons, voltages, and digital numbers, is established. Based on the proposed model, the photoresponse characteristics and signal error rates of ideal and realistic QISs are investigated. The effects of the ADC bit depth n, quantum efficiency QE, the mean value of dark signal electrons μ_d, and electron-referred read noise u_n to them are analyzed. The signal error rate between photons or photoelectrons and digital numbers SER_ph and SER_phe are defined as the quality factors of photon and photoelectron counting. When SER_ph and SER_phe are lower than 0.01, the high accuracy photon and photoelectron counting exposure ranges, which are referred as R_ph and R_phe, are determined. The main approach of extending R_ph and R_phe is increasing n and QE, and decreasing μ_d and u_n. Moreover, an integration time optimization method to ensure that the QIS works within R_ph and R_phe is presented. The trade-off between jot area A_jot, the mean value of incident photons μ_ph, and integration time τ under different illuminance level I_lux are analyzed. When n, A_jot, QE, μ_d, and u_n of QISs are determined, the integration time optimization method can be performed to guide the design and operation of QISs.

This work was supported by the Tianjin Key Laboratory of Imaging and Sensing Microelectronic Technology.