• Acta Optica Sinica
  • Vol. 40, Issue 1, 0111012 (2020)
Xiangyang Ji*
Author Affiliations
  • Department of Automation, Tsinghua University, Beijing 100084, China
  • show less
    DOI: 10.3788/AOS202040.0111012 Cite this Article Set citation alerts
    Xiangyang Ji. Coded Photography[J]. Acta Optica Sinica, 2020, 40(1): 0111012 Copy Citation Text show less
    References

    [1] Sendra-Nadal E. Carbonell-Barrachina Á A. Sensory and aroma marketing[M]. The Netherlands: Wageningen Academic Publishers(2017).

    [2] Mait J N. A history of imaging: revisiting the past to chart the future[J]. Optics and Photonics News, 17, 22-27(2006).

    [3] Mait J N, Euliss G W, Athale R A. Computational imaging[J]. Advances in Optics and Photonics, 10, 409-483(2018).

    [4] Janesick J R. Scientific charge-coupled devices[M]. Bellingham, WA: SPIE(2001).

    [5] Cooley J W, Tukey J W. An algorithm for the machine calculation of complex Fourier series[J]. Mathematics of Computation, 19, 297-301(1965).

    [7] Dowski E R, Cathey W T. Single-lens single-image incoherent passive-ranging systems[J]. Applied Optics, 33, 6762-6773(1994).

    [8] Dowski E R, Cathey W T. Extended depth of field through wave-front coding[J]. Applied Optics, 34, 1859-1866(1995).

    [9] Adelson E[M]. H, Bergen J. R. The plenoptic function and the elements of early vision(1991).

    [10] Lohmann A W, Dorsch R G, Mendlovic D et al. Space-bandwidth product of optical signals and systems[J]. Journal of the Optical Society of America A, 13, 470-473(1996).

    [11] Neifeld M A. Information, resolution, and space-bandwidth product[J]. Optics Letters, 23, 1477-1479(1998).

    [12] Zheng G A, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy[J]. Nature Photonics, 7, 739-745(2013).

    [13] Tian L, Waller L. 3D intensity and phase imaging from light field measurements in an LED array microscope[J]. Optica, 2, 104-111(2015).

    [14] Zhang Y B, Cui Z, Ji X Y et al. 3D Fourier ptychographic microscopy based on the beam propagation method and time-reversal scheme[J]. IEEE Access, 7, 129402-129410(2019).

    [15] Levin A, Fergus R, Durand F et al. Image and depth from a conventional camera with a coded aperture[J]. ACM Transactions on Graphics, 26, 70(2007).

    [16] Gehm M E, John R, Brady D J et al. Single-shot compressive spectral imaging with a dual-disperser architecture[J]. Optics Express, 15, 14013-14027(2007).

    [17] Lin J Y, Lin X, Ji X Y et al. Separable coded aperture for depth from a single image[J]. IEEE Signal Processing Letters, 12, 1471-1475(2014).

    [18] Katz O, Heidmann P, Fink M et al. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations[J]. Nature Photonics, 8, 784-790(2014).

    [19] Popoff S, Lerosey G, Fink M et al. Image transmission through an opaque material[J]. Nature Communications, 1, 81(2010).

    [20] Wang X Y, Jin X, Li J Q et al. Prior-information-free single-shot scattering imaging beyond the memory effect[J]. Optics Letters, 44, 1423-1426(2019).

    [21] Park J H, Hong K, Lee B. Recent progress in three-dimensional information processing based on integral imaging[J]. Applied Optics, 48, H77-H94(2009).

    [22] Levoy M, Ng R, Adams A et al. Light field microscopy[J]. ACM Transactions on Graphics, 25, 924-934(2006).

    [23] Raskar R, Agrawal A, Tumblin J. Coded exposure photography: motion deblurring using fluttered shutter[J]. ACM Transactions on Graphics, 25, 795-804(2006).

    [24] Gu J W, Hitomi Y, Mitsunaga T et al. Coded rolling shutter photography: flexible space-time sampling. [C]∥2010 IEEE International Conference on Computational Photography (ICCP), March 29-30, 2010, Cambridge, MA, USA. New York: IEEE, 11553751(2010).

    [25] Wan P F, Tan J X, Lian X C et al. High bit-depth image acquisition framework using embedded quantization bias[J]. IEEE Transactions on Computational Imaging, 5, 556-569(2019).

    [26] Qi X, Guo X, Harris J G. A time-to-first spike CMOS imager. [C]∥2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No. 04CH37512), May 23-26, 2004, Vancouver, BC, Canada. New York: IEEE, 824-827(2004).

    [27] Lichtsteiner P, Posch C, Delbruck T. A 128×128 120 dB 15 μs latency asynchronous temporal contrast vision sensor[J]. IEEE Journal of Solid-State Circuits, 43, 566-576(2008).

    [28] Faulkner H M L, Rodenburg J M. Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm[J]. Physical Review Letters, 93, 023903(2004).

    [29] Fienup J R. Phase retrieval algorithms: a comparison[J]. Applied Optics, 21, 2758-2769(1982).

    [30] Ou X Z, Horstmeyer R, Yang C et al. Quantitative phase imaging via Fourier ptychographic microscopy[J]. Optics Letters, 38, 4845-4848(2013).

    [31] Tian L, Waller L. Quantitative differential phase contrast imaging in an LED array microscope[J]. Optics Express, 23, 11394-11403(2015).

    [32] Guo K K, Dong S Y, Nanda P et al. Optimization of sampling pattern and the design of Fourier ptychographic illuminator[J]. Optics Express, 23, 6171-6180(2015).

    [33] Tian L, Li X, Ramchandran K et al. Multiplexed coded illumination for Fourier Ptychography with an LED array microscope[J]. Biomedical Optics Express, 5, 2376-2389(2014).

    [34] van Roey J, van der Donk J, Lagasse P E. Beam-propagation method: analysis and assessment[J]. Journal of the Optical Society of America, 71, 803-810(1981).

    [35] Sidorenko P, Cohen O. Single-shot ptychography[J]. Optica, 3, 9-14(2016).

    [36] Gustafsson M G L. Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution[J]. Proceedings of the National Academy of Sciences of the United States of America, 102, 13081-13086(2005).

    [37] Schermelleh L, Carlton P M, Haase S et al. Subdiffraction multicolor imaging of the nuclear periphery with 3D structured illumination microscopy[J]. Science, 320, 1332-1336(2008).

    [38] Goodman J W. Introduction to Fourier optics[M]. 3rd ed. Greenwood Village, Colorado: Roberts and Company Publishers(2005).

    [39] Machine Intelligence. PAMI-[J]. Pentland A P. A new sense for depth of field. IEEE Transactions on Pattern Analysis, 9, 523-531(1987).

    [40] Chakrabarti A, Zickler T. Depth and deblurring from a spectrally-varying depth-of-field[M]. ∥Fitzgibbon A, Lazebnik S, Perona P, et al. Computer vision-ECCV 2012. Lecture notes in computer science. Berlin, Heidelberg: Springer, 7576, 648-661(2012).

    [41] Haim H, Elmalem S, Giryes R et al. Depth estimation from a single image using deep learned phase coded mask[J]. IEEE Transactions on Computational Imaging, 4, 298-310(2018).

    [42] Golay M J E. Multi-slit spectrometry[J]. Journal of the Optical Society of America, 39, 437-444(1949).

    [43] Ibbett R N, Aspinall D, Grainger J F. Real-time multiplexing of dispersed spectra in any wavelength region[J]. Applied Optics, 7, 1089-1093(1968).

    [44] Gehm M E. McCain S T, Pitsianis N P, et al. Static two-dimensional aperture coding for multimodal, multiplex spectroscopy[J]. Applied optics, 45, 2965-2974(2006).

    [45] Candes E J, Romberg J, Tao T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information[J]. IEEE Transactions on Information Theory, 52, 489-509(2006).

    [46] Mooney J M, Vickers V E, An M et al. High-throughput hyperspectral infrared camera[J]. Journal of the Optical Society of America A, 14, 2951-2961(1997).

    [47] Descour M R, Volin C E, Dereniak E L et al. Demonstration of a high-speed nonscanning imaging spectrometer[J]. Optics Letters, 22, 1271-1273(1997).

    [48] Wagadarikar A, John R, Willett R et al. Single disperser design for coded aperture snapshot spectral imaging[J]. Applied Optics, 47, B44-B51(2008).

    [49] Arce G R, Brady D J, Carin L et al. Compressive coded aperture spectral imaging: an introduction[J]. IEEE Signal Processing Magazine, 31, 105-115(2013).

    [50] Wagadarikar A A, Pitsianis N P, Sun X et al. Video rate spectral imaging using a coded aperture snapshot spectral imager[J]. Optics Express, 17, 6368-6388(2009).

    [51] Cao X, Du H, Tong X et al. A prism-mask system for multispectral video acquisition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 2423-2435(2011).

    [52] Ji X Y, Miao C L, Zhang Y B et al. Separating reflective and fluorescent components for dynamic scenes[J]. Optics Communications, 404, 11-17(2017).

    [53] Rueda H, Fu C, Lau D L et al. Single aperture spectral+ToF compressive camera: toward hyperspectral+depth imagery[J]. IEEE Journal of Selected Topics in Signal Processing, 11, 992-1003(2017).

    [54] Rueda H, Arguello H, Arce G R. DMD-based implementation of patterned optical filter arrays for compressive spectral imaging[J]. Journal of the Optical Society of America A, 32, 80-89(2015).

    [55] Marcia R F, Willett R M. Compressive coded aperture superresolution image reconstruction. [C]∥2008 IEEE International Conference on Acoustics, Speech and Signal Processing, March 31-April 4, 2008, Las Vegas, NV, USA. New York: IEEE, 833-836(2008).

    [56] Vijayakumar A, Rosen J. Interferenceless coded aperture correlation holography-a new technique for recording incoherent digital holograms without two-wave interference[J]. Optics Express, 25, 13883-13896(2017).

    [57] Brady D J, Mrozack A. MacCabe K, et al. Compressive tomography[J]. Advances in Optics and Photonics, 7, 756-813(2015).

    [58] Asif M S, Ayremlou A, Sankaranarayanan A et al. FlatCam: thin, lensless cameras using coded aperture and computation[J]. IEEE Transactions on Computational Imaging, 3, 384-397(2017).

    [59] Vellekoop I M, Mosk A P. Focusing coherent light through opaque strongly scattering media[J]. Optics Letters, 32, 2309-2311(2007).

    [60] Vellekoop I M, Lagendijk A, Mosk A P. Exploiting disorder for perfect focusing[J]. Nature Photonics, 4, 320-322(2010).

    [61] van Putten E G, Akbulut D, Bertolotti J et al. Scattering lens resolves sub-100 nm structures with visible light[J]. Physical Review Letters, 106, 193905(2011).

    [62] Choi Y, Yang T D, Fang-Yen C et al. Overcoming the diffraction limit using multiple light scattering in a highly disordered medium[J]. Physical Review Letters, 107, 023902(2011).

    [63] Feng S C, Kane C, Lee P A et al. Correlations and fluctuations of coherent wave transmission through disordered media[J]. Physical Review Letters, 61, 834-837(1988).

    [64] Freund I, Rosenbluh M, Feng S C. Memory effects in propagation of optical waves through disordered media[J]. Physical Review Letters, 61, 2328-2331(1988).

    [65] Bertolotti J, van Putten E G, Blum C et al. Non-invasive imaging through opaque scattering layers[J]. Nature, 491, 232-234(2012).

    [66] Yilmaz H, van Putten E G, Bertolotti J et al. Speckle correlation resolution enhancement of wide-field fluorescence imaging[J]. Optica, 2, 424-429(2015).

    [67] Popoff S M, Lerosey G, Carminati R et al. Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media[J]. Physical Review Letters, 104, 100601(2010).

    [68] Liutkus A, Martina D, Popoff S et al. Imaging with nature: compressive imaging using a multiply scattering medium[J]. Scientific Reports, 4, 5552(2015).

    [69] Yoon J, Lee K, Park J et al. Measuring optical transmission matrices by wavefront shaping[J]. Optics Express, 23, 10158-10167(2015).

    [70] Drémeau A, Liutkus A, Martina D et al. Reference-less measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques[J]. Optics Express, 23, 11898-11911(2015).

    [71] Metzler C A, Sharma M K, Nagesh S et al. Coherent inverse scattering via transmission matrices: efficient phase retrieval algorithms and a public dataset. [C]∥2017 IEEE International Conference on Computational Photography (ICCP), May 12-14, 2017, Stanford, CA, USA. New York: IEEE, 16969263(2017).

    [72] Sharma M, Metzler C A, Nagesh S et al. Inverse scattering via transmission matrices: broadband illumination and fast phase retrieval algorithms[J]. IEEE Transactions on Computational Imaging, 1(2019).

    [73] Butterweck H J. General theory of linear, coherent optical data-processing systems[J]. Journal of the Optical Society of America, 67, 60-70(1977).

    [74] Lee K, Park Y. Exploiting the speckle-correlation scattering matrix for a compact reference-free holographic image sensor[J]. Nature Communications, 7, 13359(2016).

    [75] Mudry E, Belkebir K, Girard J et al. Structured illumination microscopy using unknown speckle patterns[J]. Nature Photonics, 6, 312-315(2012).

    [76] Antipa N, Kuo G, Heckel R et al. DiffuserCam: lensless single-exposure 3D imaging[J]. Optica, 5, 1-9(2018).

    [77] Ng R, Levoy M, Brédif M et al. Light field photography with a hand-held plenoptic camera[J]. Computer Science Technical Report CSTR, 2, 1-11(2005).

    [78] Levoy M, Hanrahan P. Light field rendering. [C]∥Proceedings of the 23rd annual conference on Computer graphics and interactive techniques-SIGGRAPH '96, August 4-9, 1996, New Orleans, LA, USA. New York: ACM, 31-42(1996).

    [79] Bishop T E, Favaro P. The light field camera: extended depth of field, aliasing, and superresolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 972-986(2012).

    [80] Bishop T E, Zanetti S, Favaro P. Light field superresolution. [C]∥2009 IEEE International Conference on Computational Photography (ICCP), April 16-17, 2009, San Francisco, CA, USA. New York: IEEE, 11499060(2009).

    [81] Fahringer T W, Lynch K P, Thurow B S. Volumetric particle image velocimetry with a single plenoptic camera[J]. Measurement Science and Technology, 26, 115201(2015).

    [82] Prevedel R, Yoon Y G, Hoffmann M et al. Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy[J]. Nature Methods, 11, 727-730(2014).

    [83] Wilburn B, Joshi N, Vaish V et al. High performance imaging using large camera arrays[J]. ACM Transactions on Graphics, 24, 765-776(2005).

    [84] Lin X, Wu J M, Zheng G A et al. Camera array based light field microscopy[J]. Biomedical Optics Express, 6, 3179-3189(2015).

    [85] Broxton M, Grosenick L, Yang S et al. Wave optics theory and 3-D deconvolution for the light field microscope[J]. Optics Express, 21, 25418-25439(2013).

    [86] Kemeny S E, Panicacci R, Pain B et al. Multiresolution image sensor[J]. IEEE Transactions on Circuits and Systems for Video Technology, 7, 575-583(1997).

    [87] Shechtman E, Caspi Y, Irani M. Increasing space-time resolution in video[M]. ∥Heyden A, Sparr G, Nielsen M, et al. Computer vision-ECCV 2002. Lecture notes in computer science. Berlin, Heidelberg: Springer, 2350, 753-768(2002).

    [88] Shechtman E, Caspi Y, Irani M. Space-time super-resolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 531-545(2005).

    [89] Portz T, Zhang L, Jiang H. Random coded sampling for high-speed HDR video. [C]∥IEEE International Conference on Computational Photography (ICCP), April 19-21, 2013, Cambridge, MA, USA. New York: IEEE, 13580210(2013).

    [90] Antipa N, Oare P, Bostan E et al. Video from stills: lensless imaging with rolling shutter. [C]∥2019 IEEE International Conference on Computational Photography (ICCP), May 15-17, 2019, Tokyo, Japan. New York: IEEE, 18793743(2019).

    [91] Attneave F. Some informational aspects of visual perception[J]. Psychological Review, 61, 183-193(1954).

    [92] Barlow H B. Sensory mechanisms, the reduction of redundancy, and intelligence[M]. London: Her Majesty’s Stationary Office, 535-539(1959).

    [93] Barlow H B, Kaushal T P, Mitchison G J. Finding minimum entropy codes[J]. Neural Computation, 1, 412-423(1989).

    [94] Perkel D H, Bullock T H. Neural coding[J]. Neurosciences Research Program Bulletin, 3, 221-348(1968).

    [95] Foldiak P[M]. Sparse coding in the primate cortex(2003).

    [96] Vinje W E. Sparse coding and decorrelation in primary visual cortex during natural vision[J]. Science, 287, 1273-1276(2000).

    [97] Bell A J, Sejnowski T J. An information-maximization approach to blind separation and blind deconvolution[J]. Neural Computation, 7, 1129-1159(1995).

    [98] Stone J V[M]. Independent component analysis: a tutorial introduction(2004).

    [99] Linsker R. Self-organization in a perceptual network[J]. Computer, 21, 105-117(1988).

    [100] Atick J J, Redlich A N. Towards a theory of early visual processing[J]. Neural Computation, 2, 308-320(1990).

    [101] Rao R P N, Ballard D H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects[J]. Nature Neuroscience, 2, 79-87(1999).

    [102] Srinivasan M V, Laughlin S B, Dubs A. Predictive coding: a fresh view of inhibition in the retina[J]. Proceedings of the Royal Society of London Series B Biological Sciences, 216, 427-459(1982).

    [103] Bialek W. Rieke F, van Steveninck R D R, et al. Reading a neural code[J]. Science, 252, 1854-1857(1991).

    [104] Berry M J, Warland D K, Meister M. The structure and precision of retinal spike trains[J]. Proceedings of the National Academy of Sciences of the United States of America, 94, 5411-5416(1997).

    [105] McKeown M J, Makeig S, Brown G G et al. Analysis of fMRI data by blind separation into independent spatial components[J]. Human Brain Mapping, 6, 160-188(1998).

    [106] van Hateren J H, van der Schaaf A. Independent component filters of natural images compared with simple cells in primary visual cortex[J]. Proceedings of the Royal Society of London Series B: Biological Sciences, 265, 359-366(1998).

    [108] Meister M. Berry M J II. The neural code of the retina[J]. Neuron, 22, 435-450(1999).

    [109] Olshausen B, Field D. Sparse coding of sensory inputs[J]. Current Opinion in Neurobiology, 14, 481-487(2004).

    [110] Zaghloul K A, Boahen K. A silicon retina that reproduces signals in the optic nerve[J]. Journal of Neural Engineering, 3, 257-267(2006).

    [111] Gollisch T, Meister M. Rapid neural coding in the retina with relative spike latencies[J]. Science, 319, 1108-1111(2008).

    [112] Boahen K A. Point-to-point connectivity between neuromorphic chips using address events[J]. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 47, 416-434(2000).

    [113] Zaghloul K A, Boahen K A. An on-off log domain circuit that recreates adaptive filtering in the retina[J]. IEEE Transactions on Circuits and Systems I: Regular Papers, 52, 99-107(2005).

    [114] Roska B, Molnar A, Werblin F S. Parallel processing in retinal ganglion cells: how integration of space-time patterns of excitation and inhibition form the spiking output[J]. Journal of Neurophysiology, 95, 3810-3822(2006).

    [115] Mahowald M. An analog VLSI system for stereoscopic vision[M]. Boston, MA: Springer(1994).

    [116] Ruedi P F, Heim P, Kaess F et al. A 128×128 pixel 120-dB dynamic-range vision-sensor chip for image contrast and orientation extraction[J]. IEEE Journal of Solid-State Circuits, 38, 2325-2333(2003).

    [117] Luo Q, Harris J G. A time-based CMOS image sensor. [C]∥2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No. 04CH37512), May 23-26, 2004, Vancouver, BC, Canada. New York: IEEE, 840-843(2004).

    [118] Azadmehr M, Abrahamsen J P, Hafliger P. A foveated AER imager chip [address event representation]. [C]∥2005 IEEE International Symposium on Circuits and Systems, May 23-26, 2005, Kobe, Japan. New York: IEEE, 2751-2754(2005).

    [119] Costas-Santos J, Serrano-Gotarredona T, Serrano-Gotarredona R et al. A spatial contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems[J]. IEEE Transactions on Circuits and Systems I: Regular Papers, 54, 1444-1458(2007).

    [120] Posch C, Matolin D, Wohlgenannt R. A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS[J]. IEEE Journal of Solid-State Circuits, 46, 259-275(2011).

    [121] Brandli C, Berner R, Yang M et al. A 240×180 130 dB 3 μs latency global shutter spatiotemporal vision sensor[J]. IEEE Journal of Solid-State Circuits, 49, 2333-2341(2014).

    [122] Li C H, Brandli C, Berner R et al. Design of an RGBW color VGA rolling and global shutter dynamic and active-pixel vision sensor. [C]∥2015 IEEE International Symposium on Circuits and Systems (ISCAS), May 24-27, 2015, Lisbon, Portugal. New York: IEEE, 718-721(2015).

    [123] Son B, Suh Y, Kim S et al. A 640×480 dynamic vision sensor with a 9 μm pixel and 300 Meps address-event representation. [C]∥2017 IEEE International Solid-State Circuits Conference (ISSCC), February 5-9, 2017, San Francisco, CA, USA. New York: IEEE, 66-67(2017).

    Xiangyang Ji. Coded Photography[J]. Acta Optica Sinica, 2020, 40(1): 0111012
    Download Citation