Ten-mega-pixel snapshot compressive imaging with a hybrid coded aperture

Zhihong Zhang; Chao Deng; Yang Liu; Xin Yuan; Jinli Suo; Qionghai Dai

doi:10.1364/PRJ.435256

[1] G. Carles, J. Downing, A. R. Harvey. Super-resolution imaging using a camera array. Opt. Lett., 39, 1889-1892(2014).

[2] S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar. Fast and robust multiframe super resolution. IEEE Trans. Image Process., 13, 1327-1344(2004).

[3] R. F. Marcia, R. M. Willett. Compressive coded aperture superresolution image reconstruction. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 833-836(2008).

[4] T. S. Cho, A. Levin, F. Durand, W. T. Freeman. Motion blur removal with orthogonal parabolic exposures. IEEE International Conference on Computational Photography (ICCP), 1-8(2010).

[5] R. Raskar, A. Agrawal, J. Tumblin. Coded exposure photography: motion deblurring using fluttered shutter. ACM Trans. Graph., 25, 795-804(2006).

[6] C. Zhou, S. Nayar. What are good apertures for defocus deblurring?. IEEE International Conference on Computational Photography (ICCP), 1-8(2009).

[7] C. Zhou, S. Lin, S. K. Nayar. Coded aperture pairs for depth from defocus and defocus deblurring. Int. J. Comput. Vis., 93, 53-72(2011).

[8] A. Sellent, P. Favaro. Optimized aperture shapes for depth estimation. Pattern Recognit. Lett., 40, 96-103(2014).

[9] S. Suwajanakorn, C. Hernandez, S. M. Seitz. Depth from focus with your mobile phone. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3497-3506(2015).

[10] P. Llull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro, D. J. Brady. Coded aperture compressive temporal imaging. Opt. Express, 21, 10526-10545(2013).

[11] A. Wagadarikar, R. John, R. Willett, D. Brady. Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt., 47, B44-B51(2008).

[12] X. Yuan, D. J. Brady, A. K. Katsaggelos. Snapshot compressive imaging: theory, algorithms, and applications. IEEE Signal Process Mag., 38, 65-88(2021).

[13] D. Reddy, A. Veeraraghavan, R. Chellappa. P2C2: programmable pixel compressive camera for high speed imaging. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 329-336(2011).

[14] Y. Hitomi, J. Gu, M. Gupta, T. Mitsunaga, S. K. Nayar. Video from a single coded exposure photograph using a learned over-complete dictionary. International Conference on Computer Vision (ICCV), 287-294(2011).

[15] M. Qiao, Z. Meng, J. Ma, X. Yuan. Deep learning for video compressive sensing. APL Photon., 5, 030801(2020).

[16] X. Yuan, P. Llull, X. Liao, J. Yang, D. J. Brady, G. Sapiro, L. Carin. Low-cost compressive sensing for color video and depth. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3318-3325(2014).

[17] X. Yuan, Y. Liu, J. Suo, Q. Dai. Plug-and-play algorithms for large-scale snapshot compressive imaging. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1444-1454(2020).

[18] X. Yuan, Y. Liu, J. Suo, F. Durand, Q. Dai. Plug-and-play algorithms for video snapshot compressive imaging(2021).

[19] Y. Sun, X. Yuan, S. Pang. Compressive high-speed stereo imaging. Opt. Express, 25, 18182-18190(2017).

[20] L. I. Rudin, S. Osher, E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 60, 259-268(1992).

[21] M. Tassano, J. Delon, T. Veit. FastDVDnet: towards real-time deep video denoising without flow estimation. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1351-1360(2020).

[22] C. Deng, Y. Zhang, Y. Mao, J. Fan, J. Suo, Z. Zhang, Q. Dai. Sinusoidal sampling enhanced compressive camera for high speed imaging. IEEE Trans. Pattern Anal. Mach. Intell., 43, 1380-1393(2021).

[23] L. Gao, J. Liang, C. Li, L. V. Wang. Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature, 516, 74-77(2014).

[24] N. Antipa, P. Oare, E. Bostan, R. Ng, L. Waller. Video from stills: lensless imaging with rolling shutter. IEEE International Conference on Computational Photography (ICCP), 1-8(2019).

[25] X. Yuan. Generalized alternating projection based total variation minimization for compressive sensing. IEEE International Conference on Image Processing (ICIP), 2539-2543(2016).

[26] Y. Liu, X. Yuan, J. Suo, D. Brady, Q. Dai. Rank minimization for snapshot compressive imaging. IEEE Trans. Pattern Anal. Mach. Intell., 41, 2990-3006(2019).

[27] J. Yang, X. Yuan, X. Liao, P. Llull, D. J. Brady, G. Sapiro, L. Carin. Video compressive sensing using Gaussian mixture models. IEEE Trans. Image Process., 23, 4863-4878(2014).

[28] J. Yang, X. Liao, X. Yuan, P. Llull, D. J. Brady, G. Sapiro, L. Carin. “Compressive sensing by learning a Gaussian mixture model from measurements. IEEE Trans. Image Process., 24, 106-119(2015).

[29] J. Bioucas-Dias, M. Figueiredo. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process., 16, 2992-3004(2007).

[30] Z. Cheng, A. Vedaldi, R. Lu, H. Bischof, T. Brox, Z. Wang, J.-M. Frahm, H. Zhang, B. Chen, Z. Meng, X. Yuan. BIRNAT: bidirectional recurrent neural networks with adversarial training for video snapshot compressive imaging. European Conference on Computer Vision, 258-275(2020).

[31] Z. Wang, H. Zhang, Z. Cheng, B. Chen, X. Yuan. MetaSCI: scalable and adaptive reconstruction for video compressive sensing. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2083-2092(2021).

[32] M. Iliadis, L. Spinoulas, A. K. Katsaggelos. Deep fully-connected networks for video compressive sensing. Digit. Signal Process., 72, 9-18(2018).

[33] K. Kulkarni, S. Lohit, P. Turaga, R. Kerviche, A. Ashok. ReconNet: non-iterative reconstruction of images from compressively sensed measurements. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 449-458(2016).

[34] J. Ma, X.-Y. Liu, Z. Shou, X. Yuan. Deep tensor ADMM-net for snapshot compressive imaging. IEEE/CVF International Conference on Computer Vision (ICCV), 10222-10231(2019).

[35] X. Liao, H. Li, L. Carin. Generalized alternating projection for weighted-l_{2, 1} minimization with applications to model-based compressive sensing. SIAM J. Imag. Sci., 7, 797-823(2014).

[36] S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations Trends Mach. Learn., 3, 1-122(2011).

[37] S. Jalali, X. Yuan. Compressive imaging via one-shot measurements. IEEE International Symposium on Information Theory, 416-420(2018).

[38] S. Jalali, X. Yuan. Snapshot compressed sensing: performance bounds and algorithms. IEEE Trans. Inf. Theory, 65, 8005-8024(2019).

[39] K. Zhang, W. Zuo, L. Zhang. FFDNet: toward a fast and flexible solution for CNN based image denoising. IEEE Trans. Image Process., 27, 4608-4622(2018).

[40] S. Gu, L. Zhang, W. Zuo, X. Feng. Weighted nuclear norm minimization with application to image denoising. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2862-2869(2014).

[41] A. Mercat, M. Viitanen, J. Vanne. UVG dataset: 50/120 fps 4K sequences for video codec analysis and development. 11th ACM Multimedia Systems Conference, 297-302(2020).