Fiber-optic imaging systems (FOISs) [1] play a critical role in clinical practice and biological research, such as the early detection of gastrointestinal cancers [2–5] and visualization of neuronal activities in freely moving animals [6–8], because of their small sizes and flexibility. To date, most FOISs have been cohesive multicore fiber bundles (CFBs) [9–13] consisting of thousands of fibers, each transmitting a single pixel. In recent years, imaging techniques based on a single multimode fiber (MMF) have attracted extensive research interest [14–29]. Compared with CFBs, an MMF supports the simultaneous propagation of over 100 optical modes through a hair-thin imaging probe, reducing the invasiveness in imaging deep tissues and organs, which gives MMF-based imaging systems great potential in the field of medical endoscopy. However, the incident beam’s original information is distorted because of mode coupling and interference, resulting in a complete scrambling of the light field in the output plane. This means that, after the coherent light is propagated through the MMF, a random speckle pattern is formed at the exit end. Moreover, the imaging resolution is constrained by the number of MMF’s inherent optical modes. The MMF imaging quality is a serious issue, especially for a small-core MMF. Therefore, reconstructing the original high-quality images from random speckle patterns in an ultrafine MMF is a significant challenge.

The widely employed approaches for reconstructing detection images in current MMF imaging systems are transmission matrix (TM)-based methods and deep learning (DL)-based methods. Because the TM describes the input–output response of an arbitrary optical system by connecting the free modes at the input plane to those at the output plane, the TM-based method is a powerful tool for directly recovering images from speckle patterns when the TM can be accurately measured [30,31]. By searching for the correct TM under the MMF’s different bending states with spatial-frequency beacon tracking, the dynamic change and interference arising from the bending of MMFs can be effectively addressed [32], enabling MMF-based endoscopy imaging with high stability and robustness against the movement and deformation of the probe. On the other hand, deep learning has shown great promise in MMF-based optical endoscopy because of rapid advances in artificial intelligence technology. DL-based methods can learn the complex nonlinear relationship inside the MMF to provide an MMF imaging technique with sustainability and robustness [23,33–38]. Driven by large amounts of data, DL-based methods can also achieve better reconstruction performance. Nevertheless, TM-based and DL-based methods are still inevitably constrained by the MMF’s inherent physical transmission characteristics. The uncontrollable spatial modes during propagation limit imaging performance. To unleash the full potential of MMF imaging, modulating the internal spatial modes of MMF is a promising method.

In the current paper, we propose a mode modulation method based on the SVD of the MMF’s TM, which could effectively enhance the imaging quality of MMF-based imaging systems. We first obtain a set of associated singular vectors by performing SVD of the MMF’s TM and then modulate the internal optical modes by displaying the two-dimensional (2D) maps of the singular vectors as a modulation on the spatial light modulator (SLM). Since the singular vectors couple the light field into eigenchannels during transmission [39–42] and reflect the transfer characteristics of the MMF, the modulated light beam exhibits extraordinary transmissivity during transmission, allowing for more efficient energy transfer, thus contributing to enhanced imaging quality. Moreover, modulating the modes by different singular vectors enables selective excitation of the MMF’s internal optical modes. Via this process, we can customize speckle patterns with the desired properties (grain size and intensity). Optimal imaging-enhanced performance can be achieved by choosing the optimal distribution of the excited modes. Based on experiments on 40 μm and 105 μm core MMF systems, we customized the properties of the distal output speckle pattern successfully with mode modulation. Furthermore, to demonstrate that mode modulation has a key impact on imaging quality, we conducted image reconstruction experiments on the 40 μm and 105 μm core MMF systems. The deep learning approach was utilized for image reconstruction [36]. The MNIST dataset [43], QuickDraw dataset [44], SIPaKMeD dataset [45], and ImageNet dataset [46] were chosen as the reconstructed objects. The experimental results showed that remarkable imaging-enhancement performance was achieved with our method across different scene types. Especially for the ultrafine 40 μm core MMF system, the image reconstruction accuracy achieved up to a 7.32 dB increase in peak signal-to-noise ratio (PSNR) and a 0.103 increase in structural similarity (SSIM) when applying the proposed mode modulation method. The imaging ability of the 40 μm core MMF system with mode modulation could outperform that of the 105 μm core MMF system without mode modulation. Our scheme can effectively promote the application of an ultrafine MMF in imaging, which is of great value in the field of low-injury medical endoscopy.

An MMF is a type of complex scattering medium in optics due to modal dispersion. Although speckle field formation is random, the beam propagation process in a static scattering medium can be determined [47]. Therefore, once the correlation characteristics of the proximal end and distal end of the MMF are measured, the transmission characteristics of the fiber can be quantitatively expressed. Therefore, the input–output response of an MMF can be described by TM.

The TM, $T$, links the input light field $E_{\mathrm{in}}$ to the output light field $E_{\mathrm{out}}$, as follows: $$E_{\mathrm{out}}={\mathit{TE}}_{\mathrm{in}}.$$(1)

Dividing the input light field and the output light field into $N$ and $M$ parts, respectively, the $m$th output light field can be expressed as follows: $$E_m^{\mathrm{out}}={\sum }_n^N{t}_{mn}{E}_n^{\mathrm{in}},$$(2)where $E_n^{\mathrm{in}}$ is the $n$th input light field and $t_{mn}$ is a TM element.

Here, we use the four-step phase-shift method [48] to obtain the TM of the MMF.

If the phase of the light varies with a constant $\alpha$, the intensity of the $m$th output mode is given by the following: $$\begin{gathered} I_m^{\alpha }={\left|E_m^{\mathrm{out}}\right|}^2={\left|s_m+\sum _n^N{\mathrm{e}}^{i\alpha }{t}_{mn}{E}_n^{\mathrm{in}}\right|}^2 \\ ={\left|s_m\right|}^2+{\left|\sum _n^N{\mathrm{e}}^{i\alpha }{t}_{mn}{E}_n^{\mathrm{in}}\right|}^2+2\Re \left({\mathrm{e}}^{i\alpha }{\overline{s}}_m\sum _n^N{t}_{mn}{E}_n^{\mathrm{in}}\right), \end{gathered}$$(3)where $s_m$ is the complex amplitude of the optical field used as the reference in the $m$th output mode.

Thus, the intensities in the $m$th output mode $I_m^{\alpha }$ for $\alpha =0$, $\pi /2$, $\pi$, and $3\pi /2$ are obtained by injecting the $n$th input mode with phase shift of $\alpha$. The measured intensities $I_m^0$, $I_m^{\pi /2}$, $I_m^{\pi }$, and $I_m^{3\pi /2}$ give $$\frac{(I_m^0-I_m^{\pi })}{4}+i\frac{(I_m^{3\pi /2}-I_m^{\pi /2})}{4}={\overline{s}}_m{t}_{mn}{E}_n^{\mathrm{in}}.$$(4)

The observed TM embedded with the reference light can be measured by selecting the appropriate orthogonal input matrix. Here, we chose the Hadamard basis as the input matrix. We obtain the intensities of the output light fields corresponding to a set of input light fields. Then, we introduce the parameter into Eq. (4). All the $t_{mn}$ elements are solved to form the TM, $T$.

The mode modulation of the MMF is achieved based on the singular vectors and their corresponding singular values, thus exciting the internal optical modes of the MMF selectively and customizing the properties of the output speckle. To give access to the singular vectors, we performed the SVD of the measured TM.

The SVD of a matrix $T_{(m,n)}$ (size $m\times n$) consists of a decomposition such that $$T_{(m,n)}=U_{(m,m)}{\mathrm{\Sigma }}_{(m,n)}{V}_{(n,n)}^{†},$$(5)where $\mathrm{\Sigma }$ is a rectangular diagonal matrix with non-negative real numbers on the diagonal called singular values $\sigma$. The singular value ${\sigma }_i$ was sorted in descending order of the column index of $\mathrm{\Sigma }$, with the column index defined as an eigenchannel index $i$. $V$ and $U$ are the unitary matrices containing the so-called right and left singular vectors, the columns of which are the transmission eigenchannels at the input and output planes, respectively. The singular values, $\sigma$, are the transmission coefficients of these eigenchannels [41]. $V^{†}$ denotes the conjugate transpose of matrix $V$.

As in the SVD definition, when illuminating the MMF with an input singular vector (e.g., the $i$th, $V_i$), the transmission is driven by the associated singular value: $T{V}_i={\sigma }_i{U}_i$. Because of the singular vectors’ unitarity, the total intensity is given by the following: $$I=V_i^{†}{T}^{†}T{V}_i={\sigma }_i^2{U}_i^{†}{U}_i={\sigma }_i^2.$$(6)

As the singular vectors return the eigenmodes of the TM associated with specific values of transmission, they result in specificity in spatial frequency [49]. Different singular vector numbers specifically excite the different orders of modes in the MMF. The schematic overview is shown in Fig. 1. Each column of $V$ is converted into a discrete 2D map as the input light field displayed on the SLM. Highly transmitting singular vectors (singular vectors corresponding to high singular values) concentrate light in low-order spatial modes of low frequency, resulting in enlarged speckle grain sizes and high intensity. Conversely, singular vectors with lower transmission (singular vectors corresponding to relatively low singular values) tend to excite high-order spatial modes of high frequency in the MMF, leading to smaller speckle grain sizes and lower intensity. During the eigenchannel transmission, the transmittance of the singular vectors $V_i$ decreases with the increase of $i$. So, theoretically, the grain size and intensity of the speckle patterns at the distal end of the MMF will gradually decrease with an increase $i$ of the singular vector $V_i$. Furthermore, the singular vectors inject the light beam into the eigenchannels for transmission. This injection is beneficial for information transmission through an MMF as the eigenchannels have extraordinarily high transmission and can transmit energy more efficiently than other transmission channels [41]. Thus, the light beam modulated by singular vectors demonstrates extraordinary transmission efficiency, which is advantageous for information transmission.

The contour features of the images are carried by low-order spatial frequencies, and the fine features of the images are carried by higher-order spatial frequencies that correspond to the low- and high-order modes in the MMF, respectively. Theoretically, a large number of excited optical modes, especially higher-order modes, are advantageous for the high-fidelity reconstruction of images [24].

During information transmission, the internal energy loss of MMF is inevitable. In general, the low-order spatial mode possesses relatively low energy loss, while the high-order spatial mode possesses higher energy loss. The increased energy loss results in a degree of decrease in average speckle intensity, which is manifested in the low brightness of the light-intensity region. It is worth noting that extremely low speckle intensity will lead to information loss.

To sum up, higher-order modes favor the high-fidelity reconstruction of images. Nevertheless, the excitation of higher-order modes leads to low speckle intensity—that is, the loss of image information. As a result, an optimal distribution of the excited modes is essential for the quality of the MMF imaging system. This distribution should ensure that both the grain size and intensity of the corresponding output speckle are appropriately suited for MMF imaging purposes.

The experimental setup is shown in Fig. 2, here containing two processes. The first process was the acquisition of the MMF’s TM. The second process was MMF imaging with mode modulation.

TM acquisition was set as follows: a laser beam of 632.8 nm wavelength was generated by a He-Ne laser (Thorlabs, HNL210LB) and then expanded by a beam expander (BE). The laser beam was split into an object beam and a reference beam using a polarizing beam splitter (PBS1). The object beam reflected a set of orthogonal Hadamard basis displayed by a reflective phase-only spatial light modulator (SLM, $1920\times 1080$, $6.4\mathrm{\mu m}\times 6.4\mathrm{\mu m}$, FSLM-2K55-P02, CAS MICROSTAR). The object beam was then focused onto the step-index MMF facet (20 cm length, 40 μm/105 μm core diameter, NA = 0.22, YOFC) using a microscope objective (OBJ1, PLN $40\times$, Olympus). The generated focus was coupled into the fiber’s propagation modes, outputted at the other facet, and magnified by OBJ2 (same as OBJ1). The object beam transmitted through the MMF was recombined with the reference beam using another polarizing beam splitter (PBS2), forming the corresponding interference speckle patterns, which were captured by a camera ($1280\times 1024$ pixels, Basler ace acA1300-200um).

The MMF imaging with mode modulation was set as follows: the expanded laser beam from the He-Ne laser illuminated the digital micromirror device (DMD, $1024\times 768$, $13.7\mathrm{\mu m}\times 13.7\mathrm{\mu m}$, V-7000, VIALUX), which loaded the object input patterns that were controlled by a computer. The beam with object information was then modulated by the SLM, which was driven by a selected singular vector 2D map. The modulated light beam was transmitted through the MMF and then collected by the camera. This modulation with SLM enabled capturing the desired speckle patterns at the camera.

For the system that used the 40 μm core MMF to propagate and evolve different transverse modes, we obtained 289 singular vectors by performing the SVD of the measured TM. To quantitatively compare the speckle grain size for different singular vectors, we calculated the autocorrelation of the output pattern and extracted its full width at half maximum (FWHM) to obtain the grain size for each speckle pattern. Figure 3(a) shows the speckle grain size as a function of the singular vector number. The red dots represent the corresponding 30 selected singular vectors that are used to customize the speckle properties for the 40 μm core MMF imaging testing. These dots are not equidistant as the speckle field formation is random [47]. We selected the 30 singular vectors such that the spacing between the speckle grain size corresponding to each selected singular vector is as uniform as possible. Accordingly, the speckle intensity also varied with the singular vector number. The corresponding relationship is shown in Fig. 3(b). As mentioned above, the results have indicated that the grain size and intensity of the speckle patterns at the distal end of MMF were followed by a downward trend, with an increase in the singular vector number. To visually verify the effectiveness of customizing the speckle properties with mode modulation, we recorded the light field at the image plane of the speckle patterns with the largest and smallest speckle grain size, respectively. As observed in Fig. 3(c), the high-transmission singular vector (No. 3) resulted in an enlarged speckle grain size when compared with the low-transmission singular vector (No. 279). We also calculated the Fourier transform of these speckle patterns to visualize the distribution of spatial frequencies. As shown in Fig. 3(d), the No. 3 singular vector corresponded to a narrower $k$ distribution in Fourier space compared with that of the No. 279 singular vector. Figure 3(e) shows the central cuts through the two Fourier space distributions. The $k$ distribution of the No. 3 singular vector peaked at around $|k|=0$, meaning that the spatial modes with low frequency were transmitted better than the high-frequency ones. For the low transmission singular vector (No. 279), the $k$ distribution in Fourier space peaked at a relatively high $|k|$ value. The results have indicated that the high-frequency components were transmitted more efficiently.

We effectively achieved the selective excitation of the internal optical modes of MMF through mode modulation. Different distributions of excited modes led to variations in the output speckle’s properties. Furthermore, we have also validated the effectiveness of speckle properties customization through mode modulation on a 105 μm core MMF system (see Appendix A). As shown in the results of both MMF systems, the speckle grain size and intensity were followed by a downward trend with an increase in the singular vector number. By selecting the singular vector within this range for mode modulation, we could achieve the desired distribution of the excited modes, thus obtaining speckle patterns with the desired speckle properties. Notably, this range for mode modulation is determined by the number of singular vectors obtained through the SVD of the measured TM.

Here, the MNIST, QuickDraw, SIPaKMeD, and ImageNet datasets were chosen as the reconstructed objects. We resized all images of these datasets to a consistent dimension of $48\times 48$ pixels. We used the FC–encoder–decoder network [36] to recover the input images from the random speckle patterns. We collected 10,000 data pairs for each dataset, with 9000 data pairs selected as the training dataset and 1000 data pairs as the testing dataset. We collected a total of 31 sets of output speckle patterns, here consisting of the corresponding output speckle patterns obtained with mode modulation by 30 selected singular vectors, together with the reference speckle patterns obtained without mode modulation. In our experiments, the speckle patterns captured by the 40 μm core MMF system and the 105 μm core MMF system were of sizes $\text{340\hspace{0.17em}\hspace{0.17em}}\times 340$ and $\text{900\hspace{0.17em}\hspace{0.17em}}\times 900$, respectively. When training the network, we resized all the collected output speckle patterns to $1\text{92\hspace{0.17em}\hspace{0.17em}}\times 192$ pixels. The image reconstruction results of the 105 μm core MMF system are presented in Appendix B. The SSIM and PSNR were selected as the objective evaluation index of image reconstruction accuracy.

The experimental results of the MNIST, QuickDraw, SIPaKMeD, and ImageNet datasets on the 40 μm core MMF system are shown in Fig. 4. We present both the reconstruction results of the optimal singular vector and inferior singular vector simultaneously. The optimal singular vector refers to the singular vector that obtained the optimal imaging-enhancement performance after applying it for mode modulation, while the inferior singular vector refers to the singular vector that obtained the worst imaging-enhancement performance after applying it for mode modulation. Ref represents the reconstruction results without mode modulation. No. 3, No. 10, No. 16, No. 147, No. 160, No. 217, and No. 238 represent the reconstruction results with mode modulation by the singular vectors of the corresponding number. Compared with the reconstruction results obtained without mode modulation, the image reconstruction accuracy of these datasets improved with mode modulation by singular vectors.

For the MNIST dataset, the No. 160 singular vector showed the greatest imaging-enhancement performance, with an increase of 7.32 dB in PSNR and 0.071 in SSIM. The No. 10 singular vector showed the lowest imaging-enhancement performance, with an increase of 4.84 dB in PSNR and 0.046 in SSIM. For the QuickDraw dataset, the No. 217 singular vector showed the greatest imaging-enhancement performance, with an increase of 6.16 dB in PSNR and 0.103 in SSIM. The No. 16 singular vector showed the lowest imaging-enhancement performance, with an increase of 3.32 dB in PSNR and 0.055 in SSIM. For the SIPaKMeD dataset, the No. 238 singular vector showed the greatest imaging-enhancement performance, with an increase of 1.83 dB in PSNR and 0.046 in SSIM. The No. 3 singular vector showed the lowest imaging-enhancement performance, with an increase of 1.19 dB in PSNR and 0.027 in SSIM. For the ImageNet dataset, the No. 147 singular vector showed the greatest imaging-enhancement performance, with an increase of 1.2 dB in PSNR and 0.064 in SSIM. The No. 3 singular vector showed the lowest imaging-enhancement performance, with an increase of 0.75 dB in PSNR and 0.031 in SSIM.

Without mode modulation, the reconstructed images from the MNIST, QuickDraw, and SIPaKMeD datasets appeared to be blurred. The reconstructed complex natural images from the ImageNet dataset only had a partially reconstructed contour, and the image features were barely distinguishable by the human eye. Apparently, with mode modulation, image reconstruction performance was significantly improved. The images from the MNIST dataset and QuickDraw dataset achieved high-quality recovery. Note that the cell images from the SIPaKMeD dataset and natural scene images from the ImageNet dataset were grayscale, containing more information and being much harder to reconstruct. However, with mode modulation, the imaging quality was visually significantly improved, and the reconstructed images could already be clearly recognized.

Based on 30 sets of reconstruction results, we analyzed the reconstruction performance with different singular vectors. As shown in Fig. 5(a), the relationship between image reconstruction accuracy and speckle grain size can be described using a parabolic curve. To conduct a quantitative analysis, we calculated the autocorrelation and extracted the FWHM of the speckle patterns obtained with mode modulation by the optimal singular vectors for each dataset. As shown in Fig. 5(b), the optimal speckle grain sizes for the MNIST, QuickDraw, SIPaKMeD, and ImageNet datasets were 7.18 pixels, 7.05 pixels, 6.96 pixels, and 7.27 pixels, respectively. The corresponding speckle intensities for the MNIST, QuickDraw, SIPaKMeD, and ImageNet datasets were 13.7, 9.61, 9.56, and 11.83, respectively. We achieved the optimal distribution of the excited modes with mode modulation to strike a balance between speckle grain size and intensity, thus obtaining the best-performing speckle pattern for reconstruction. Because of the varying frequency information distributions across different scenes, the optimal distribution of excited modes in the MMF correspondingly differed. Therefore, the optimal singular vector was distinct for each dataset.

We compared the reconstruction results of the ImageNet dataset on MMF systems with two different core diameters. As shown in Fig. 6(a), the imaging quality of each dataset was effectively improved with mode modulation in both the 40 μm and 105 μm core MMF systems. Notably, the reconstruction results of the 105 μm core MMF system were sharper than those of the 40 μm core MMF system and showed better reconstruction of details.

Regardless of mode modulation, the 105 μm core MMF system exhibited higher reconstruction accuracy compared with the 40 μm core MMF system. As shown in Fig. 6(b), the 105 μm core MMF produced denser speckle grains with smaller sizes compared with the 40 μm core MMF. The superior performance of the 105 μm core MMF in image reconstruction can be attributed to the presence of more high-order optical modes. Apparently, fibers with larger core diameters carry more information [25]. The number of optical modes of MMF is given by $N\sim {(\pi a\cdot \mathrm{N}\mathrm{A}/\lambda )}^2$, where $a$ is the radius of the core, NA is the numerical aperture of the fiber, and $\lambda$ is the wavelength of transmitted light [50]. Based on the calculations, the number of modes at 632.8 nm for the 40 μm core MMF is $\approx 477$ and $\approx 3288$ for the 105 μm core MMF.

In practical endoscopic applications, the utilization of finer MMF offers the advantage of reduced invasiveness. However, this may be accompanied by a decrease in resolution. With the mode modulation method, the imaging quality could be effectively improved on an ultrafine MMF. As shown in Fig. 6(a), the imaging quality of the 40 μm core MMF system with mode modulation surpassed that of the 105 μm core MMF system without mode modulation. This outcome highlights the significant advantage offered by our proposed mode modulation method for ultrafine MMF. Besides, our proposed mode modulation method also offers promise for single-shot endoscopic imaging [29] through an ultrafine MMF.

In the present paper, we have proposed a mode modulation method based on SVD for MMF imaging enhancement. We obtained a set of singular vectors by performing the SVD of the TM. The 2D maps of singular vectors were then displayed on the SLM for mode modulation. In this way, we achieved the selective excitation of the MMF internal optical modes, as demonstrated by the variance of the properties (grain size and intensity) of the output speckle patterns. By conducting image reconstruction experiments on four types of datasets, we demonstrated that image reconstruction accuracy was effectively improved with mode modulation. In particular, we could choose the optimal singular vector for mode modulation and maximize imaging-enhancement performance. As a result, the maximum improvement of image reconstruction accuracy reached up to 7.32 dB in PSNR and 0.103 in SSIM when conducting the image reconstruction experiment on the 40 μm core MMF system. Compared with the former research on MMF imaging, we experimentally developed the excellent information transmission capability of the singular vectors and fully unlocked the imaging potential of MMF. The optimal distribution of MMF’s excited modes during transmission was explored and realized. It is worth mentioning that, even for the ultrafine 40 μm core MMF with limited mode quantity, the reconstructed images could be of a high resolution with mode modulation, surpassing that of the 105 μm core MMF system without mode modulation. Thus, our method has presented a fresh avenue for qualitatively enhancing the imaging quality of the ultrafine MMF imaging system in noninvasive medicine.