When the laser is transmitted in the multimode fiber (MMF), speckle patterns with randomly varied bright and dark regions are formed at the output end of the fiber due to the influence of the interference, coupling, and dispersion among the modes. In the field of imaging, these speckles are typically considered as noise and are minimized as much as possible. However, in sensing technology, the speckle phenomenon is actively utilized^[1–3]. MMFs can convert subtle changes in the measurands into detectable variations in the output speckle patterns, thereby enabling precise monitoring of these parameters^[4,5]. Fiber specklegram sensors (FSSs) operate on this principle. However, the high nonlinearity and sensitivity of MMFs^[6] make it difficult to demodulate measurands from speckle patterns using mathematical models. Currently, traditional methods for implementing FSSs mainly involve statistical analysis and deep learning. Although statistical analysis can quantify pixel changes in speckle patterns^[7,8], these statistical functions are prone to saturation, and their sensing range and accuracy are limited due to their reliance on linear fitting.

Deep learning, by training models to recognize measurands within speckle patterns, possesses a powerful capability for nonlinear approximation^[9,10]. However, many existing studies simply apply deep learning without an in-depth discussion of the underlying physical phenomena, which leads to a key issue: as the number of modes in MMFs increases, the normalized frequency grows, and the decorrelation rate of the speckle pattern accelerates, leading to a decrease in sensing accuracy^[11,12]. This phenomenon is attributed to the increased sensitivity of the speckle to the changes in the measurands. As more modes are supported by the fiber, differences between the texture of unknown speckle patterns and those in the training set will be more significant, which requires a method that can effectively extract the feature of the speckle patterns. On the other hand, although increasing the sampling density of the training dataset can help capture variations between speckles, it undoubtedly adds to the workload of data acquisition and the complexity of data processing. Therefore, relying solely on deep learning is insufficient to efficiently and accurately demodulate the measurands from speckle patterns. Moreover, regression models in deep learning often use pooling layers without parameters for downsampling, which not only extends the training time but may also lead to the loss of critical information, potentially introducing random errors in the sensing process.

To address the aforementioned challenges, this study proposes an innovative method based on digital aperture filtering (DAF), which adopts physical principle analysis and incorporates deep learning techniques as a data processing tool. Based on the fiber mode coupling theory^[13], the speckle pattern can be represented as a linear superposition of the fiber’s modes. By taking advantage of the physical distribution characteristic that higher-order modes have greater energy density near the core-cladding interface compared to lower-order modes, a virtual DAF with varying radii can be constructed to selectively screen out the energy from high-order modes in the speckle pattern^[14,15]. This process effectively enhances the correlation among speckle patterns and numerically compresses the fiber’s numerical aperture (NA). Subsequently, a multi-layer convolutional neural network (MLCNN) is developed accordingly, which employs convolutional layers to replace the pooling layers to achieve feature extraction and downsampling of speckles simultaneously^[16]. To validate the superiority of the DAF method in the sensing performance of MMFs, we specifically focus on experimental studies on light field direction sensing with various MMFs as a pilot study. The results demonstrate that the DAF method can significantly enhance the overall sensing performance of MMFs, including sensing accuracy, sensing range, stability, resolution, and generalizability. Notably, even when the light field detection range is doubled, the sensing accuracy of MMFs remains comparable to a few-mode fiber (FMF), which tends to be more accurate, but its applications are limited by the specific equipment.

The schematic diagram of the light field direction sensing system is depicted in Fig. 1(a). Considering the influence of the wavelength on the number of modes and the commonly used wavelength in practical applications, we select a He–Ne laser with a wavelength of 633 nm. The laser beam is modulated to circular polarization by a polarizer and a quarter-wave plate (QWP) since it is better maintained during transmission in the fiber^[17]. Subsequently, the beam is directed onto a mirror, and then the reflected light is received by the fiber. A microscope objective then projects the speckle emitted from the fiber onto a detector (Hikrobot MV-CB120-10UM-B, $4032\times 3036$ resolution with the pixel size of $1.85\mathrm{\mu m}\times 1.85\mathrm{\mu m}$). Additionally, the light field direction $\theta$ is controlled by an electric rotation stage (Zolix RAP100, minimum resolution 0.00125°) mounted beneath the mirror. To verify that the images received by the detector are speckles at the output end of the fiber, an additional LED light path is incorporated to the experimental system^[9]. In the experiment, three fibers with different modes are utilized: FMF 1 (0.12 NA), MMF 1 (0.22 NA), and MMF 2 (0.22 NA), with core diameters of 9, 62.5, and 105 µm, and they respectively approximately support 14, 2300, and 6500 modes at $\lambda =633\mathrm{nm}$. Based on the NA calculation formula for fiber: $\mathrm{NA}=n\times \sin \alpha$, where $n$ is the refractive index of the medium ($n=1$ in the air) and $\alpha$ is the light field detection angle that can be transmitted in the fiber, it can be determined that FMF 1 has a detection range of $\pm 6°$, while MMF 1 and MMF 2 offer a broader detection range, extending to $\pm 13°$.

The electric rotation stage is utilized to change the incident light field direction to the fiber within the NA range of each fiber, with a step size of 0.1° in the $\theta$ direction, which helps us to fully capture the fiber’s response to changes in the light field direction. A total of 50 speckle images are collected for each light field direction to cover the influence of the perturbation, and multiple data collection processes are used to improve the diversity and robustness of data. After collecting the speckle patterns, the speckle information from the fiber core region is extracted.

Considering the effect of coupling efficiency, the intensity of the captured speckle patterns decreases as the light field direction increases. Standardization processing is applied to the speckle patterns to ensure that the intensity values of speckles are within the range of [0, 255]. Figure 1(b) presents the speckles produced by different fibers under specific incident light field directions. Intriguingly, as the light field direction increases, the speckle energy distribution in MMFs gradually shifts from the core center toward the core-cladding interface, making the speckle distribution more complex and granular. This phenomenon is attributed to the increased light field direction, which leads to an increase in beam angular misalignment located in the fiber core and, consequently, excites a greater number of higher-order modes. Particularly, MMF 2, with its larger core diameter, supports a larger number of higher-order modes, thereby exhibiting a more complex spatial distribution of speckles. Figure 1(c) illustrates the variation of the speckle pattern’s correlation coefficient with changes in the light field direction. The correlation coefficient is obtained by collecting 30 sets of data and statistical analysis to ensure the accuracy and reliability of the results. The decorrelation angle is defined as the angular displacement of $\mathrm{\Delta }\theta$ in the light field direction. The correlation of the speckle patterns begins to diminish gradually and asymptotically approaches a stable state^[8,9]. To provide a more accurate observation of determining the size of the decorrelation angle corresponding to different fibers, we have included a solid line in the figure as a visual assistance. It is distinctly observable from Fig. 1(c) that FMF 1 possesses the largest decorrelation angle, whereas MMF 2 exhibits the smallest. This trend is in concordance with the findings reported in the current literature. It also indicates that FMF 1 exhibits a stronger correlation among its speckles, which confers superior sensing accuracy for the light field direction compared to MMF 1 and MMF 2.

To enhance the accuracy of light field direction sensing for MMF 1 and MMF 2, it is essential to mitigate the impact of higher-order modes on speckle correlation and to increase the decorrelation angle. Leveraging the energy distribution characteristics of higher-order modes, we innovatively propose to construct a virtual mask to perform DAF on the speckles from MMF 1 and MMF 2, and calculate the variation of the speckles’ normalized total intensity with respect to the light field direction $\theta$ across various digital aperture filter radii (DAFRs) R, which constitute the angular point spread functions (PSFs), in order to assess the compression effect on the fiber’s NA after DAF. As depicted in Figs. 2(a) and 2(b), the angular PSFs have a sharper angular distribution, signifying an effective compression in the NA. Furthermore, we analyze the correlation coefficient of the filtered speckle patterns with the light field direction, thereby calculating the corresponding decorrelation angles to determine the optimal DAFRs. The results are presented in Figs. 2(c) and 2(d), with the decorrelation angles illustrated in the upper right sub-figures. The curves indicate that the decorrelation angle initially increases and then decreases with the reduction of the DAFR. This trend is potentially due to the fact that the initial reduction in DAFR primarily filters out the energy of higher-order modes, thus enhancing the speckles’ correlation. During this phase, the decorrelation angle gradually increases. However, a further reduction in DAFR leads to an excessive proportion of lower-order modes and a gradual decrease in the effective feature information carried by the speckles. This over-filtering ultimately results in an excessively high correlation coefficient among speckles under different incident light field directions, causing the correlation coefficient to rapidly reach a stable state even with changes in the light field direction. Subsequently, according to the influence of the decorrelation angle on the sensing accuracy described above, it is imperative to select a DAFR that corresponds to a larger decorrelation angle. Based on the above analysis, the optimal DAFR for MMF 1 is $R=62.5/(2\times 2.7)\mathrm{\mu m}$ and for MMF 2 is $R=105/(2\times 3)\mathrm{\mu m}$. The detailed determination steps are in Sec. 3 in the Supplement 1. Additionally, to verify the correctness of the optimal DAFR selection, we have further conducted comparative analyses, with the specific data, are presented in Sec. 7 in the Supplement 1.

To effectively extract the light field direction from the filtered speckles, this study employs deep learning due to the inherent nonlinear effects of fibers. Traditional regression models frequently employ pooling layers for downsampling, which undoubtedly affects the efficiency of training. To overcome this limitation, we introduce an innovative approach that utilizes convolutional layers for feature extraction and downsampling simultaneously, enhancing training efficiency without compromising accuracy. The detailed architecture of the MLCNN is presented in Fig. 3. The network primarily consists of 10 convolutional (Conv) layers of size $3\times 3$, one $2\times 2$ Maxpooling (Pool) layer, and two fully connected (FC) layers. Each convolutional layer (stride = 2) is followed by a leaky rectified linear unit (LeakyReLU) and a batch normalization (BN) layer. The MLCNN training parameters are detailed in Sec. 4 in the Supplement 1.

Subsequently, the speckle patterns processed by DAF [MMF 1, $R=62.5/(2\times 2.7)\mathrm{\mu m}$], [MMF 2, $R=105/(2\times 3)\mathrm{\mu m}$] are shuffled and divided into training and validation sets for the MLCNN in a 4:1 ratio to ensure the diversity and effectiveness of the MLCNN training, and the corresponding light field direction is taken as the output of MLCNN. The sizes of the MLCNN datasets are detailed in Sec. 5 in the Supplement 1. Compared to the traditional networks that use pooling layers for downsampling, the MLCNN achieved a 37.5% reduction after 50 epochs. To comprehensively evaluate the performance of the MLCNN, we construct two separate test sets. Test set I includes the speckle patterns independently captured to correspond to light field directions within the training set range, with a $\theta$ direction range from $-13°$ to 13°, at a step size of 0.5°, primarily used to assess the accuracy of the trained MLCNN in light field direction sensing. Test set II includes the speckle patterns independently acquired to correspond to light field directions outside the training set range, with a $\theta$ direction range from $-12.75°$ to 12.75°, at a step size of 0.5°, mainly used to evaluate the generalization ability of the MLCNN to unknown speckles. For each light field direction, 30 independent datasets are collected. To verify the effectiveness of the DAF method, we compare the sensing results with those of the unfiltered speckle demodulation (uFSD) method. In addition, comparisons are also made with FMF 1 based on the uFSD method to further evaluate the performance advantages of our proposed DAF method.

Figure 4 shows the comparison of experimental results for different methods in the light field direction sensing based on MMF 1. To facilitate a more intuitive analysis, the graph presents the absolute error between the perceived direction and the true direction. The central data point represents the mean absolute error (MAE) of 30 datasets of sensing results, which serves as a benchmark for measuring sensing accuracy. The blue and yellow points correspond to the sensing results of the DAF method and the uFSD method, respectively. The red points denote the sensing results of FMF 1 based on the uFSD method. The shaded area represents the standard deviation (S.D.) of the data, and its width indicates the degree of dispersion of the sensing results, which signifies the repeatability and stability of the sensing results.

From the visual representation, it is evident that the uFSD method exhibits larger errors when sensing large-angle light field directions. In contrast, the DAF method not only maintains high accuracy within a small angle range but also significantly enhances sensing performance across larger angles. This is reflected in the smaller fluctuations of the MAE curve, indicating the advantage of the DAF method in sensing accuracy and range. Furthermore, the area of the shaded region has been shrunk, demonstrating the superiority of the DAF method in improving the stability of results. It is noteworthy that the fluctuation amplitude of the MAE curve for the DAF method is similar to the MAE curve of FMF 1, further corroborating the effectiveness of the DAF method in improving the light field direction sensing performance of MMFs.

Figure 5 presents a comparative analysis of performance among different methods in the light field direction sensing based on MMF 2. The results demonstrate that although the MAE curve for MMF 2 exhibits greater fluctuations compared to MMF 1, the MAE curve based on the DAF method shows significantly less fluctuation than that based on the uFSD method, particularly for the large angle light field direction sensing. This finding underscores that, even when the proposed DAF method is applied to MMF 2, which has a higher number of modes, more complex speckle patterns, and lower correlation, it can still achieve a significant enhancement in sensing accuracy, sensing range, and stability. This not only confirms the universality of the DAF method across different MMFs but also accentuates its pronounced effectiveness and practical value in improving sensing capabilities.

Additionally, to provide a comprehensive assessment of the DAF method’s sensing performance, the sensing results are quantified using MAE, S.D., and maximum absolute error (MaxAE), as shown in Table 1. Then, we conduct a thorough comparison with the uFSD method in terms of sensing accuracy, sensing range, stability, resolution, and generalizability. In sensing accuracy, the DAF method significantly reduces the MAE, with a reduction of approximately 50%. Particularly for MMF 1, even with the light field direction sensing range doubled, its sensing accuracy remains comparable to that of FMF 1. Regarding the sensing range, the DAF method effectively minimizes the sensing errors of the uFSD method for large-angle light field directions, thereby expanding the sensing range. In terms of stability, the S.D. of the sensing results obtained by the DAF method is also less than that of the uFSD method, indicating better sensing consistency. Regarding resolution, we define the resolution based on the $3\sigma$ rule, where $\sigma$ is the S.D., and the results show that the resolution of the DAF method is superior to that of the uFSD method. Moreover, by comparing the sensing error gaps between two test sets, we find that the error gap between the test sets based on the DAF method is smaller, which further demonstrates that the DAF method has better consistency and generalizability across different datasets. In summary, the proposed DAF method is superior to the uFSD method in all aspects, confirming the effectiveness of the DAF method. The detailed analysis can be found in Sec. 6 in the Supplement 1.

In conclusion, to address the performance degradation issue of deep learning-based FSSs when processing speckles from MMFs, an innovative DAF method based on physical principle analysis is proposed. This method utilizes the differences in energy density distribution between higher-order and lower-order modes to effectively screen out the influence of higher-order modes. This provides optimized input data for the MLCNN, which has superior training efficiency compared to the traditional regression model. Through performance tests on light field direction sensing with different MMFs, the results demonstrate the significant advantage of the DAF method over the uFSD method in sensing performance. Compared to the uFSD method, the DAF method achieves a reduction of approximately 50% in sensing error. Moreover, the DAF method effectively reduced sensing errors at large-angle light field directions, successfully expanding the sensing range. Regarding stability, the S.D. of the sensing results from the DAF method is 55% lower than that of the uFSD method, indicating the reliability of the DAF method in providing more consistent sensing results. For resolution, the value of the DAF method is about 62% smaller than that of the uFSD method. Additionally, the smaller error gaps between test sets based on the DAF method prove their better generalizability under different conditions.

In summary, our experimental studies have confirmed the effectiveness of the proposed DAF method in mitigating the impact of higher-order modes and enhancing the sensing performance of MMFs, suggesting its prospective applicability across different measurands, such as strain, displacement, and bending curvature, heralding a significant advancement in the domain of FSSs. At the same time, we recognize that enhancing anti-interference capabilities is a challenge faced by the FSS field in general, especially under extreme conditions. Therefore, we plan to focus on improving the sensor’s anti-interference ability as a key direction for future research.