Optical imaging typically relies on a camera with millions of photodetectors. By contrast, imaging can also take place with a single detector. This is particularly advantageous for imaging at non-visible spectrum where the conventional pixelated cameras lose the sensitivity or become very costly for a good performance.

A most straightforward way to image through a single detector is to scan the object and measure it pixel by pixel. Yet, another interesting approach, commonly known as single-pixel camera, is to leverage compressed sensing. Instead of being sampled pixel by pixel, the object is sampled through the sum of different combinations of pixels. Typically, the object is illuminated with a pattern, and the detector captures the sum intensity of the object pixels illuminated by the pattern. By performing the measurement through different patterns, the entire object can be reconstructed.

In both the conventional camera and the point-scanning-based system, the actual number of measurements is the same as the total number of pixels, though the former measures all pixels at once through different detectors and the latter measures individual pixel sequentially through a single detector. However, the number of measurements, i.e. the number of illumination patterns, can be much smaller than the number of pixels (for example 10~100 times smaller) in compressed sensing. The computation algorithm can then recover the object in high quality through a small number of measurements, by exploiting the natural relationship between adjacent pixels. As a result, compressed sensing has a very high measurement efficiency. The fewer measurements make it promising to reduce the data acquisition time and thus increase the imaging speed.

However, the existing implementation of compressed sensing in imaging relies on switching different illumination masks, and the limited speed in the switching device (digital mirror device) restraints the overall speed. In many scenarios, the imaging speed of the switching-mask-based single-pixel camera is actually lower than that based on point scanning or conventional cameras.

The research group led by Prof. Weijian Yang from University of California, Davis recently demonstrated a new compressed imaging modality, termed "deep compressed imaging via optimized pattern scanning" (DeCIOPS), which could overcome the challenges in the existing single-pixel cameras, and greatly improve the imaging speed of compressed imaging. The research results are published in Photonics Research, Vol. 9, No. 3, 2021 (Kangning Zhang, Junjie Hu, Weijian Yang. Deep compressed imaging via optimized pattern scanning[J]. Photonics Research, 2021, 9(3): 03000B57 ).

Schematics of DeCIOPS (deep compressed imaging via optimized pattern scanning). The object is sampled block by block by an illumination pattern. ISTA-Net, a compressed-sensing-inspired deep neural network is used to reconstruct the object. The illumination pattern and the parameters in ISTA-Net are optimized through an end-to-end trained auto-encoder.

In this new imaging modality (DeCIOPS), a block of pixels, weighted by an illumination pattern, are sampled and summed into a single measurement. By scanning the illumination pattern across the object, different blocks of pixels could be measured. A reconstruction algorithm based on a compressed-sensing-inspired deep neural network was developed to recover the high resolution image from this seemingly low-resolution block-by-block measurement. Since the number of measurement blocks can be much smaller than the number of total pixels, the measurement speed can be very high.

In addition, the researchers modeled DeCIOPS through an auto-encoder, which was trained in an end-to-end manner and jointly optimized the illumination pattern for image acquisition and the deep neural network for object reconstruction. The end-to-end optimization ensures an overall optimal performance of the entire imaging system and thus a high quality in object reconstruction.

Essentially, DeCIOPS synthesizes the strengths of the high acquisition speed of point scanning and the high measurement efficiency of compressed sensing. Compared to the point scanning system, DeCIOPS increases the imaging speed (by a few folds to one order of magnitude) through a highly efficient sampling scheme. Compared to the conventional compressed imaging which requires switching between many illumination patterns, DeCIOPS scans a single illumination pattern across the object, and could increase the overall acquisition speed by orders of magnitude.

"DeCIOPS represents a nice example in computational imaging with a tight integration of the imaging system and the computation algorithm," said Prof. Weijian Yang. He believes that the high imaging speed offered in DeCIOPS makes it very promising for applications in biomedicine, surveillance, and consumer electronics.