Multi-Camera System: Imaging Enhancement and Application

Peiyao Guo; Zhiyuan Pu; Zhan Ma

doi:10.3788/LOP202158.1811013

[1] Brady D J, Pang W B, Li H et al. Parallel cameras[J]. Optica, 5, 127-137(2018).

[2] Dai Q H, Suo J L, Ji X Y et al. Computational photography: computational capture of plenoptic visual information[M](2016).

[3] Barbastathis G, Ozcan A, Situ G H. On the use of deep learning for computational imaging[J]. Optica, 6, 921-943(2019).

[4] Cao X, Yue T, Lin X et al. Computational snapshot multispectral cameras: toward dynamic capture of the spectral world[J]. IEEE Signal Processing Magazine, 33, 95-108(2016).

[5] Manekar R, Tayal K, Kumar V et al. End-to-end learning for phase retrieval[C]. //ICML workshop on ML Interpretability for Scientific Discovery, July 13-18, 2020, Virtual Event(2020).

[6] Yao Y, Zuo M, Dong Y et al. Polarization imaging feature characterization of different endometrium phases by machine learning[J]. OSA Continuum, 4, 1776-1791(2021).

[7] Suo J L, Liu Y B, Ji X Y et al. Computational photography: keys, methods and applications[J]. Acta Automatica Sinica, 41, 669-685(2015).

[8] Adelson E H, Bergen J R[M]. // Landy M, Movshon J A. Computational models of visual processing(1991).

[9] Zuo C, Feng S J, Zhang X Y et al. Deep learning based computational imaging: status, challenges, and future[J]. Acta Optica Sinica, 40, 0111003(2020).

[10] Shao X P, Liu F, Li W et al. Latest progress in computational imaging technology and application[J]. Laser & Optoelectronics Progress, 57, 020001(2020).

[11] Fang L, Dai Q H. Computational light field imaging[J]. Acta Optica Sinica, 40, 0111001(2020).

[12] Nomura Y, Zhang L, Nayar S K. Scene collages and flexible camera arrays[C]. //Proceedings of the 18th Eurographics Conference on Rendering Techniques, May 25, 2007, Grenoble, France, 127-138(2007).

[13] Lin X, Liu Y B, Wu J M et al. Spatial-spectral encoded compressive hyperspectral imaging[J]. ACM Transactions on Graphics, 33, 1-11(2014).

[14] Land M F, Nilsson D E. Lens eyes on land[M]. //Animal eyes, 94-129(2012).

[15] Mildenhall B, Srinivasan P P, Tancik M et al. NeRF: representing scenes as neural radiance fields for view synthesis[M]. //Vedaldi A, Bischof H, Brox T, et al. Computer vision-ECCV 2020. Lecture notes in computer science, 405-421(2020).

[17] Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision[C]. //IJCAI’81: Proceedings of the 7th International Joint Conference on Artificial Intelligence, August 24-28, 1981, Vancouver, BC, Canada(1981).

[18] de Tone D, Malisiewicz T, Rabinovich A. Deep image homography estimation[EB/OL]. (2016-06-13)[2021-05-25]. https://arxiv.org/abs/1606.03798

[19] Dai J F, Qi H Z, Xiong Y W et al. Deformable convolutional networks[C]. //2017 IEEE International Conference on Computer Vision (ICCV), October 22-29, 2017, Venice, Italy., 764-773(2017).

[20] Liu Y, Chen X, Wang Z F et al. Deep learning for pixel-level image fusion: recent advances and future prospects[J]. Information Fusion, 42, 158-173(2018).

[21] Kaur H, Koundal D, Kadyan V. Image fusion techniques: a survey[J]. Archives of Computational Methods in Engineering, 1-23(2021).

[22] Li S T, Kang X D, Hu J W. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing, 22, 2864-2875(2013).

[23] Lohmann A W. Scaling laws for lens systems[J]. Applied Optics, 28, 4996-4998(1989).

[24] Kopf J, Uyttendaele M, Deussen O et al. Capturing and viewing gigapixel images[J]. ACM Transactions on Graphics, 26, 93(2007).

[25] Brady D J, Gehm M E, Stack R A et al. Multiscale gigapixel photography[J]. Nature, 486, 386-389(2012).

[26] Zhang J N, Zhu T Y, Zhang A K et al. Multiscale-VR: multiscale gigapixel 3D panoramic videography for virtual reality[C]. //2020 IEEE International Conference on Computational Photography (ICCP), April 24-26, 2020, St. Louis, MO, USA, 1-12(2020).

[27] Wilburn B, Joshi N, Vaish V et al. High performance imaging using large camera arrays[J]. ACM Transactions on Graphics, 24, 765-776(2005).

[28] Ding C Q, Ma Z. Multi-camera color correction via hybrid histogram matching[EB/OL]. [2020-05-31]. https://box.nju.edu.cn/f/ef598961b2bf4c1a8dae/

[29] Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 60, 91-110(2004). http://doi.ieeecomputersociety.org/resolve?ref_id=doi:10.1023/B:VISI.0000029664.99615.94&rfr_id=trans/tp/2008/10/ttp2008101683.htm

[30] Cossairt O S, Miau D, Nayar S K. Gigapixel computational imaging[C]. //2011 IEEE International Conference on Computational Photography (ICCP), April 8-10, 2011, Pittsburgh, PA, USA., 1-8(2011).

[31] Son H S, Marks D L, Hahn J et al. Design of a spherical focal surface using close-packed relay optics[J]. Optics Express, 19, 16132-16138(2011).

[32] Yuan X Y, Ji M Q, Wu J M et al. A modular hierarchical array camera[J]. Light, Science & Applications, 10, 37(2021).

[33] Yuan X Y, Fang L, Dai Q H et al. Multiscale gigapixel video: a cross resolution image matching and warping approach[C]. //2017 IEEE International Conference on Computational Photography (ICCP), May 12-14, 2017, Stanford, CA, USA, 1-9(2017).

[34] Shechtman E, Caspi Y, Irani M. Increasing space-time resolution in video[M]. //Heyden A, Sparr G, Nielsen M, et al. Computer vision-ECCV 2002. Lecture notes in computer science, 2350, 753-768(2002).

[35] Wilburn B, Joshi N, Vaish V et al. High-speed videography using a dense camera array[C]. //Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, June 27-July 2, 2004, Washington, DC, USA., II(2004).

[36] Wang T C, Zhu J Y, Kalantari N K et al. Light field video capture using a learning-based hybrid imaging system[J]. ACM Transactions on Graphics, 36, 1-13(2017).

[37] Cheng M, Ma Z, Asif S et al. A dual camera system for high spatiotemporal resolution video acquisition[J/OL]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. [2020-05-31]. https:∥ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9052471

[38] Debevec P E, Malik J. Recovering high dynamic range radiance maps from photographs[C]. //ACM SIGGRAPH 2008 classes on-SIGGRAPH’08, August 11-15, 2008, Los Angeles, California., 1-10(2008).

[39] Chen Y Y, Jiang G Y, Yu M et al. Learning stereo high dynamic range imaging from a pair of cameras with different exposure parameters[J]. IEEE Transactions on Computational Imaging, 6, 1044-1058(2020).

[40] Lee B, Song B C. Multi-image high dynamic range algorithm using a hybrid camera[J]. Signal Processing: Image Communication, 30, 37-56(2015).

[41] Froehlich J, Grandinetti S, Eberhardt B et al. Creating cinematic wide gamut HDR-video for the evaluation of tone mapping operators and HDR-displays[J]. Proceedings of SPIE, 9023, 90230X(2014).

[42] Tocci M D, Kiser C, Tocci N et al. A versatile HDR video production system[J]. ACM Transactions on Graphics, 30, 1-10(2011).

[43] Huo Y Q, Yang F, Dong L et al. Physiological inverse tone mapping based on retina response[J]. The Visual Computer, 30, 507-517(2014).

[44] Banterle F, Ledda P, Debattista K et al. Expanding low dynamic range videos for high dynamic range applications[C]. //Proceedings of the 24th Spring Conference on Computer Graphics-SCCG’08, April 21-23, 2008, Budmerice, Slovakia., 33-41(2010).

[45] Lin H Y, Chang W Z. High dynamic range imaging for stereoscopic scene representation[C]. //2009 16th IEEE International Conference on Image Processing (ICIP), November 7-10, 2009, Cairo, Egypt., 4305-4308(2009).

[46] Selmanovic E, Debattista K, Bashford-Rogers T et al. Enabling stereoscopic high dynamic range video[J]. Signal Processing: Image Communication, 29, 216-228(2014).

[47] Bätz M, Richter T, Garbas J U et al. High dynamic range video reconstruction from a stereo camera setup[J]. Signal Processing: Image Communication, 29, 191-202(2014).

[48] Park W J, Ji S W, Kang S J et al. Stereo vision-based high dynamic range imaging using differently-exposed image pair[J]. Sensors, 17, 1473(2017).

[49] Heo Y S, Lee K M, Lee S U. Depth maps and high-dynamic range image generation from alternating exposure multiview images[J]. Journal of Electronic Imaging, 26, 043006(2017).

[50] Pu Z Y, Guo P Y, Asif M S et al. Robust high dynamic range (HDR) imaging with complex motion and parallax[M]. //Ishikawa H, Liu C L, Pajdla T, et al. Computer vision-ACCV 2020. Lecture notes in computer science, 12623, 134-149(2021).

[51] Orozco R R, Loscos C, Martin I et al. HDR multiview image sequence generation[M]. //High dynamic range video, 61-86(2017).

[52] Pizer S M, Johnston R E, Ericksen J P et al. Contrast-limited adaptive histogram equalization: speed and effectiveness[C]. //Proceedings of the First Conference on Visualization in Biomedical Computing, May 22-25, 1990, Atlanta, GA, USA, 337-345(1990).

[53] Jobson D J, Rahman Z, Woodell G A. A multiscale retinex for bridging the gap between color images and the human observation of scenes[J]. IEEE Transactions on Image Processing, 6, 965-976(1997).

[54] Liu J Y, Xu D J, Yang W H et al. Benchmarking low-light image enhancement and beyond[J]. International Journal of Computer Vision, 129, 1153-1184(2021).

[55] Petschnigg G, Szeliski R, Agrawala M et al. Digital photography with flash and no-flash image pairs[J]. ACM Transactions on Graphics, 23, 664-672(2004).

[56] Krishnan D, Fergus R. Dark flash photography[J]. ACM Transactions on Graphics, 28, 1-11(2009).

[57] Wang J, Xue T F, Barron J T et al. Stereoscopic dark flash for low-light photography[C]. //2019 IEEE International Conference on Computational Photography (ICCP), May 15-17, 2019, Tokyo, Japan., 1-10(2019).

[58] Dong X, Li W X. Shoot high-quality color images using dual-lens system with monochrome and color cameras[J]. Neurocomputing, 352, 22-32(2019).

[59] Guo P Y, Ma Z. Low-light color imaging via dual camera acquisition[M]. //Ishikawa H, Liu C L, Pajdla T, et al. Computer vision-ACCV 2020. Lecture notes in computer science, 12623, 150-167(2021).

[60] Liu Y. Special research report on optics: continuous innovation of mobile camera and upgrading of multi-directional support technology[R](2020).

[61] Brady D J, Fang L, Ma Z. Deep learning for camera data acquisition, control, and image estimation[J]. Advances in Optics and Photonics, 12, 787-846(2020).