• Acta Optica Sinica
  • Vol. 40, Issue 1, 0111001 (2020)
Lu Fang and Qionghai Dai*
Author Affiliations
  • Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China
  • show less
    DOI: 10.3788/AOS202040.0111001 Cite this Article Set citation alerts
    Lu Fang, Qionghai Dai. Computational Light Field Imaging[J]. Acta Optica Sinica, 2020, 40(1): 0111001 Copy Citation Text show less
    References

    [1] Gershun A. The light field[J]. Journal of Mathematics and Physics, 18, 51-151(1939).

    [2] Moon P, Spencer D E[M]. The photic field(1981).

    [3] Adelson H, Bergen J[M]. The plenoptic function and the elements of early vision, 3-20(1991).

    [4] Levoy M, Hanrahan P. Light field rendering. [C]∥Proceedings of the 23rd annual conference on Computer graphics and interactive techniques-SIGGRAPH'96, August 4-9, 1996, New Orleans, LA, USA. New York: ACM, 31-42(1996).

    [7] Levoy M, Ng R, Adams A et al. Light field microscopy[J]. ACM Transactions on Graphics, 25, 924-934(2006).

    [8] Levoy M. Light fields and computational imaging[J]. Computer, 39, 46-55(2006).

    [9] Ihrke I, Restrepo J, Mignard-Debise L. Principles of light field imaging: briefly revisiting 25 years of research[J]. IEEE Signal Processing Magazine, 33, 59-69(2016).

    [10] Wetzstein G, Ihrke I, Lanman D et al. Computational plenoptic imaging[J]. Computer Graphics Forum, 30, 2397-2426(2011).

    [11] Chen S E. Quicktime VR: an image-based approach to virtual environment navigation. [C]∥Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, August 6-11, 1995, Los Angeles, CA, USA. New York: ACM, 29-38(1995).

    [12] McMillan L, Bishop G. Plenoptic modeling: an image-based rendering system. [C]∥Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, August 6-11, 1995, Los Angeles, CA, USA. New York: ACM, 39-46(1995).

    [13] Chai J X, Chan S C, Shum H Y et al. Plenoptic sampling. [C]∥Proceedings of the 27th annual conference on Computer graphics and interactive techniques-SIGGRAPH'00, July 23-28, 2000, New Orleans, LA, USA. New York: ACM, 307-318(2000).

    [14] Lin Z C, Shum H Y. A geometric analysis of light field rendering[J]. International Journal of Computer Vision, 58, 121-138(2004).

    [15] Buehler C, Bosse M. McMillan L, et al. Unstructured lumigraph rendering. [C]∥Proceedings of the 28th annual conference on Computer graphics and interactive techniques-SIGGRAPH'01, August 12-17, 2001, Los Angeles, California, USA. New York: ACM, 425-432(2001).

    [16] Ihrke I, Wetzstein G, Heidrich W. A theory of plenoptic multiplexing. [C]∥2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 13-18, 2010, San Francisco, CA, USA. New York: IEEE, 483-490(2010).

    [17] Wilburn B, Joshi N, Vaish V et al. High performance imaging using large camera arrays[J]. ACM Transactions on Graphics, 24, 765-776(2005).

    [18] Yang J C, Everett M, Buehler C et al. A real-time distributed light field camera. [C]∥Proceedings of the 13th Eurographics Workshop on Rendering Techniques, June 26-28, 2002, Pisa, Italy. Frontenex: Eurographics Association, 77-86(2002).

    [19] Venkataraman K, Lelescu D, Duparré J et al. PiCam: an ultra-thin high performance monolithic camera array[J]. ACM Transactions on Graphics (TOG), 32, 166(2013).

    [20] Wilburn B S, Smulski M. Lee H H K, et al. Light field video camera[J]. Proceedings of SPIE, 4674, 29-36(2001).

    [21] Wilburn B, Joshi N, Vaish V et al. High-speed videography using a dense camera array. [C]∥Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. June 27-July 2, 2004, Washington, DC, USA. New York: IEEE, 8168976(2004).

    [22] Zhang C, Chen T. A self-reconfigurable camera array. [C]∥Proceedings of the 15th Eurographics Workshop on Rendering Techniques, June 21-23, 2004, Norköping, Sweden. Frontenex: Eurographics Association, 243-254(2004).

    [23] Chan S C, Ng K T, Gan Z F et al. The plenoptic video[J]. IEEE Transactions on Circuits and Systems for Video Technology, 15, 1650-1659(2005).

    [24] Liu Y B, Dai Q H, Xu W L. A real time interactive dynamic light field transmission system. [C]∥2006 IEEE International Conference on Multimedia and Expo, July 9-12, 2006, Toronto, ON, Canada. New York: IEEE, 2173-2176(2006).

    [27] Unger J, Wenger A, Hawkins T et al. Capturing and rendering with incident light fields. [C]∥Proceedings of the 14th Eurographics workshop on Rendering Techniques, June 25-27, 2003, Leuven, Belgium. Frontenex: Eurographics Association, 141-149(2003).

    [28] Kim C, Zimmer H, Pritch Y et al. Scene reconstruction from high spatio-angular resolution light fields[J]. ACM Transactions on Graphics, 32, 73(2013).

    [29] Dansereau D G, Schuster G, Ford J et al. A wide-field-of-view monocentric light field camera. [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE, 3757-3766(2017).

    [30] Liang C K, Lin T H, Wong B Y et al. Programmable aperture photography: multiplexed light field acquisition[J]. ACM Transactions on Graphics, 27, 55(2008).

    [31] Ihrke I, Stich T, Gottschlich H et al. Fast incident light field acquisition and rendering[J]. Journal of WSCG, 16, 25-32(2008).

    [32] Taguchi Y, Agrawal A, Ramalingam S et al. Axial light field for curved mirrors: reflect your perspective, widen your view. [C]∥2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 13-18, 2010, San Francisco, CA, USA. New York: IEEE, 499-506(2010).

    [33] Lippmann G. La photographie integrale[J]. Academie des Sciences, 146, 446-451(1908).

    [34] Adelson E H. Wang J Y A. Single lens stereo with a plenoptic camera[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14, 99-106(1992).

    [35] Ng R, Levoy M, Brédif M et al. Light field photography with a hand-held plenoptic camera[R]. USA: Computer Science Technical Report CSTR(2005).

    [37] Wei L Y, Liang C K, Myhre G et al. Improving light field camera sample design with irregularity and aberration[J]. ACM Transactions on Graphics, 34, 152(2015).

    [38] Georgiev T, Lumsdaine A. Superresolution with plenoptic camera 2.0[R]. USA: Adobe Systems Incorporated(2009).

    [39] Horstmeyer R, Euliss G, Athale R. Flexible multimodal camera using a light field architecture. [C]∥2009 IEEE International Conference on Computational Photography (ICCP), April 16-17, 2009, San Francisco, CA, USA. New York: IEEE, 11499063(2009).

    [40] Levin A, Fergus R, Durand F et al. Image and depth from a conventional camera with a coded aperture[J]. ACM Transactions on Graphics, 26, 70(2007).

    [41] Manakov A, Restrepo J F, Klehm O et al. A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imaging[J]. ACM Transactions on Graphics, 32, 47(2013).

    [42] Lanman D, Crispell D, Wachs M et al. Spherical catadioptric arrays: construction, multi-view geometry, and calibration. [C]∥Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06), June 14-16, 2006, Chapel Hill, NC, USA. New York: IEEE, 10285053(2006).

    [43] Levoy M, Zhang Z. McDowall I. Recording and controlling the 4D light field in a microscope using microlens arrays[J]. Journal of Microscopy, 235, 144-162(2009).

    [44] Prevedel R, Yoon Y G, Hoffmann M et al. Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy[J]. Nature Methods, 11, 727-730(2014).

    [45] Nöbauer T, Skocek O. Pernía-Andrade A J, et al. Video rate volumetric Ca 2+ imaging across cortex using seeded iterative demixing (SID) microscopy[J]. Nature Methods, 14, 811-818(2017).

    [46] Lin X, Wu J M, Zheng G A et al. Camera array based light field microscopy[J]. Biomedical Optics Express, 6, 3179-3189(2015).

    [47] Mignard-Debise L, Ihrke I. Light-field microscopy with a consumer light-field camera. [C]∥2015 International Conference on 3D Vision, October 19-22, 2015, Lyon, France. New York: IEEE, 335-343(2015).

    [48] Georgiev T G, Zheng K C, Curless B et al. Spatio-angular resolution tradeoffs in integral photography. [C]∥Proceedings of the Eurographics Symposium on Rendering Techniques, Nicosia, Cyprus. Frontenex: Eurographics Association, 263-272(2006).

    [49] Veeraraghavan A, Raskar R, Agrawal A et al. Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing. [C]∥SIGGRAPH'07 ACM SIGGRAPH 2007 papers, August 5-9, 2007, San Diego, California. New York: ACM, 69(2007).

    [50] Lanman D, Raskar R, Agrawal A et al. Shield fields: modeling and capturing 3D occluders[J]. ACM Transactions on Graphics, 27, 131(2008).

    [51] Amit A, Mark A. Compressive light field imaging[J]. Proceedings of SPIE, 7690, 76900Q(2010).

    [52] Antipa N, Necula S, Ng R et al. Single-shot diffuser-encoded light field imaging. [C]∥2016 IEEE International Conference on Computational Photography (ICCP), May 13-15, 2016, Evanston, IL, USA. New York: IEEE(2016).

    [53] Pégard N C, Liu H Y, Antipa N et al. Compressive light-field microscopy for 3D neural activity recording[J]. Optica, 3, 517-524(2016).

    [54] Cohen N, Yang S, Andalman A et al. Enhancing the performance of the light field microscope using wavefront coding[J]. Optics Express, 22, 24817-24839(2014).

    [55] Marwah K, Wetzstein G, Bando Y et al. Compressive light field photography using overcomplete dictionaries and optimized projections[J]. ACM Transactions on Graphics, 32, 46(2013).

    [56] Wang Y P, Wang L C, Kong D H et al. High-resolution light field capture with coded aperture[J]. IEEE Transactions on Image Processing, 24, 5609-5618(2015).

    [57] Kopf J, Uyttendaele M, Deussen O et al. Capturing and viewing gigapixel images[J]. ACM Transactions on Graphics, 26, 93(2007).

    [59] Pirk S, Cohen M F, Deussen O et al. Video enhanced gigapixel panoramas. [C]∥SIGGRAPH Asia 2012 Technical Briefs on-SA'12, November 28-December 1, 2012, Singapore, Singapore. New York: ACM, 7(2012).

    [60] Cossairt O S, Miau D, Nayar S K. Gigapixel computational imaging. [C]∥2011 IEEE International Conference on Computational Photography (ICCP), April 8-10, 2011, Pittsburgh, PA, USA. New York: IEEE, 11962821(2011).

    [61] Cossairt O S, Miau D, Nayar S K. Scaling law for computational imaging using spherical optics[J]. Journal of the Optical Society of America A, 28, 2540-2553(2011).

    [62] Brady D J, Gehm M E, Stack R A et al. Multiscale gigapixel photography[J]. Nature, 486, 386-389(2012).

    [63] Kittle D S, Daniel L, David J. Automated calibration and optical testing of the AWARE-2 gigapixel multiscale camera[J]. Proceedings of SPIE, 8660, 866006(2013).

    [64] Kittle D S, Marks D L, Son H S et al. A testbed for wide-field, high-resolution, gigapixel-class cameras[J]. Review of Scientific Instruments, 84, 053107(2013).

    [65] Pang W, Brady D J. Parallel MMS gigapixel imagers. [C]∥Computational Optical Sensing and Imaging, June 26-29, 2017, San Francisco, California, United States. Washington, D. C.: OSA, CM2B, 3(2017).

    [66] Schuster G M, Agurok I P, Ford J E et al. Panoramic monocentric light field camera. [C]∥International Optical Design Conference, July 9-13, 2017, Denver, Colorado, United States. Washington, D. C.: OSA, ITh4A, 5(2017).

    [67] Wang Y W, Liu Y B, Heidrich W et al. The light field attachment: turning a DSLR into a light field camera using a low budget camera ring[J]. IEEE Transactions on Visualization and Computer Graphics, 23, 2357-2364(2017).

    [68] Yuan X Y, Fang L, Dai Q H et al. Multiscale gigapixel video: a cross resolution image matching and warping approach. [C]∥2017 IEEE International Conference on Computational Photography (ICCP), May 12-14, 2017, Stanford, CA, USA. New York: IEEE, 16969274(2017).

    [69] Wanner S, Goldluecke B. Variational light field analysis for disparity estimation and super-resolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36, 606-619(2014).

    [70] Zhang Z, Liu Y B, Dai Q H. Light field from micro-baseline image pair. [C]∥2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA. New York: IEEE, 3800-3809(2015).

    [71] Pearson J, Brookes M, Dragotti P L. Plenoptic layer-based modeling for image based rendering[J]. IEEE Transactions on Image Processing, 22, 3405-3419(2013).

    [72] Penner E, Zhang L. Soft 3D reconstruction for view synthesis[J]. ACM Transactions on Graphics, 36, 235(2017).

    [73] Chaurasia G, Duchene S, Sorkine-Hornung O et al. Depth synthesis and local warps for plausible image-based navigation[J]. ACM Transactions on Graphics, 32, 30(2013).

    [74] Kalantari N K, Wang T C, Ramamoorthi R. Learning-based view synthesis for light field cameras[J]. ACM Transactions on Graphics, 35, 193(2016).

    [75] Zhou T, Tucker R, Flynn J et al. -05-24)[2019-11-08]. https:∥arxiv.xilesou., top/abs/1805, 09817(2018).

    [76] Mildenhall B, Srinivasan P P, Ortiz-Cayon R et al. -05-02)[2019-11-08]. https:∥arxiv.xilesou., top/abs/1905, 00889(2019).

    [77] Chai J X, Ton X, Chan S C et al. Plenoptic sampling. [C]∥Proceedings of the 27th annual conference on Computer graphics and interactive techniques-SIGGRAPH'00, July 23-28, 2000, New Orleans, LA, USA. New York: ACM, 307-318(2000).

    [78] Shi L X, Hassanieh H, Davis A et al. Light field reconstruction using sparsity in the continuous Fourier domain[J]. ACM Transactions on Graphics, 34, 12(2014).

    [79] Vagharshakyan S, Bregovic R, Gotchev A. Light field reconstruction using shearlet transform[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 133-147(2018).

    [80] Vagharshakyan S, Bregovic R, Gotchev A. Accelerated shearlet-domain light field reconstruction[J]. IEEE Journal of Selected Topics in Signal Processing, 11, 1082-1091(2017).

    [81] Yoon Y, Jeon H G, Yoo D et al. Learning a deep convolutional network for light-field image super-resolution. [C]∥2015 IEEE International Conference on Computer Vision Workshop (ICCVW), December 7-13, 2015, Santiago, Chile. New York: IEEE, 57-65(2015).

    [82] Yeung H W F, Hou J H, Chen J et al. Fast light field reconstruction with deep coarse-to-fine modeling of spatial-angular clues[M]. ∥Ferrari V, Hebert M, Sminchisescu C, et al. Computer vision-ECCV 2018. Lecture notes in computer science. Cham: Springer, 11210, 138-154(2018).

    [83] Wu G C, Liu Y B, Fang L et al. Light field reconstruction using convolutional network on EPI and extended applications[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41, 1681-1694(2019).

    [84] Dai Q H, Suo J L, Ji X Y et al[M]. Computational photography: computational capture of plenoptic visual information, 11(2016).

    [85] Zhu K, Xue Y J, Fu Q et al. Hyperspectral light field stereo matching[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41, 1131-1143(2019).

    [86] Holloway J, Mitra K, Koppal S J et al. Generalized assorted camera arrays: robust cross-channel registration and applications[J]. IEEE Transactions on Image Processing, 24, 823-835(2015).

    [87] Xiong Z W, Wang L Z, Li H Q et al. Snapshot hyperspectral light field imaging. [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE, 6873-6881(2017).

    [88] Lin X, Rivenson Y, Yardimci N T et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 361, 1004-1008(2018).

    [89] Yan T, Wu J M, Zhou T K et al. Fourier-space diffractive deep neural network[J]. Physical Review Letters, 123, 023901(2019).

    Lu Fang, Qionghai Dai. Computational Light Field Imaging[J]. Acta Optica Sinica, 2020, 40(1): 0111001
    Download Citation