• Advanced Photonics
  • Vol. 6, Issue 2, 024001 (2024)
Wenhao Tang1、†, Qing Yang1、2、3, Hang Xu1, Yiyu Guo1, Jiqiang Zhang1, Chunfang Ouyang4, Leixin Meng1、*, and Xu Liu2、3、*
Author Affiliations
  • 1Zhejiang Laboratory, Research Center for Frontier Fundamental Studies, Hangzhou, China
  • 2Zhejiang University, College of Optical Science and Engineering, State Key Laboratory of Extreme Photonics and Instrumentation, Hangzhou, China
  • 3ZJU-Hangzhou Global Scientific and Technological Innovation Center, Hangzhou, China
  • 4Shanghai Jiao Tong University, Chip Hub for Integrated Photonics Xplore (CHIPX), Wuxi, China
  • show less
    DOI: 10.1117/1.AP.6.2.024001 Cite this Article Set citation alerts
    Wenhao Tang, Qing Yang, Hang Xu, Yiyu Guo, Jiqiang Zhang, Chunfang Ouyang, Leixin Meng, Xu Liu. Review of bio-inspired image sensors for efficient machine vision[J]. Advanced Photonics, 2024, 6(2): 024001 Copy Citation Text show less
    Comparison between bio-inspired and traditional image sensors.
    Fig. 1. Comparison between bio-inspired and traditional image sensors.
    Diagram of human visual information processing.
    Fig. 2. Diagram of human visual information processing.
    Bio-inspired image sensors for efficient machine vision. Adapted with permission from Refs. 2325" target="_self" style="display: inline;">–25.
    Fig. 3. Bio-inspired image sensors for efficient machine vision. Adapted with permission from Refs. 2325" target="_self" style="display: inline;">–25.
    Bio-inspired sensory architectures. (a) The overview of processing near-sensor architecture system without ADC. Reproduced with permission from Ref. 31. (b) CIM near-sensor architecture. Reproduced with permission from Ref. 32. (c) Schematic of IGZO phototransistor array to realize in-sensor compression simulation. Reproduced with permission from Ref. 33. (d) Optical enhancement and electrical suppression of IGZO phototransistors. Reproduced with permission from Ref. 33. (e) The recognition accuracy of MNIST images reconstructed with different sampling rates. Reproduced with permission from Ref. 33. (f) 2D retinomorphic device structure and motion detection of trichromatic trolleys. Reproduced with permission from Ref. 34.
    Fig. 4. Bio-inspired sensory architectures. (a) The overview of processing near-sensor architecture system without ADC. Reproduced with permission from Ref. 31. (b) CIM near-sensor architecture. Reproduced with permission from Ref. 32. (c) Schematic of IGZO phototransistor array to realize in-sensor compression simulation. Reproduced with permission from Ref. 33. (d) Optical enhancement and electrical suppression of IGZO phototransistors. Reproduced with permission from Ref. 33. (e) The recognition accuracy of MNIST images reconstructed with different sampling rates. Reproduced with permission from Ref. 33. (f) 2D retinomorphic device structure and motion detection of trichromatic trolleys. Reproduced with permission from Ref. 34.
    Novel sensory devices based on sparse coding. (a) Light stimulus-induced spike trains. Reproduced with permission from Ref. 49. (b) Image recognition using photosensor-multivibrator circuit and photonic synapse. Reproduced with permission from Ref. 49. (c) Spike-number-dependent amplitude variation of excitatory postsynaptic current (ΔEPSC) triggered by a train of optical spikes. Reproduced with permission from Ref. 49. (d) Schematic of a multifunctional memristor array stimulated by various electrical and optical inputs. Reproduced with permission from Ref. 50. (e) Read-current responses of a memristor by several optical input signals. Reproduced with permission from Ref. 50. (f) The operation of optoelectronic RC based on 2D SnS memristors for classifying consonants and vowels in the Korean alphabet. Reproduced with permission from Ref. 50.
    Fig. 5. Novel sensory devices based on sparse coding. (a) Light stimulus-induced spike trains. Reproduced with permission from Ref. 49. (b) Image recognition using photosensor-multivibrator circuit and photonic synapse. Reproduced with permission from Ref. 49. (c) Spike-number-dependent amplitude variation of excitatory postsynaptic current (ΔEPSC) triggered by a train of optical spikes. Reproduced with permission from Ref. 49. (d) Schematic of a multifunctional memristor array stimulated by various electrical and optical inputs. Reproduced with permission from Ref. 50. (e) Read-current responses of a memristor by several optical input signals. Reproduced with permission from Ref. 50. (f) The operation of optoelectronic RC based on 2D SnS memristors for classifying consonants and vowels in the Korean alphabet. Reproduced with permission from Ref. 50.
    Neural adaptation sensors for visual compression. (a) Event-driven sampling and frame-based sampling. Reproduced with permission from Ref. 51. (b) CSDVS pixel circuit. Reproduced with permission from Ref. 52. (c) Comparison of simulated normal DVS and CSDVS response to a flashing spot. Reproduced with permission from Ref. 52. (d) Illustration of a machine vision system based on the MoS2 phototransistor array. Reproduced with permission from Ref. 53. (e) Light- and dark-adapted mechanisms of the In2O3 transistor. Reproduced with permission from Ref. 54. (f) Electrical enhancement and light-depression function of an In2O3 transistor. Reproduced with permission from Ref. 54.
    Fig. 6. Neural adaptation sensors for visual compression. (a) Event-driven sampling and frame-based sampling. Reproduced with permission from Ref. 51. (b) CSDVS pixel circuit. Reproduced with permission from Ref. 52. (c) Comparison of simulated normal DVS and CSDVS response to a flashing spot. Reproduced with permission from Ref. 52. (d) Illustration of a machine vision system based on the MoS2 phototransistor array. Reproduced with permission from Ref. 53. (e) Light- and dark-adapted mechanisms of the In2O3 transistor. Reproduced with permission from Ref. 54. (f) Electrical enhancement and light-depression function of an In2O3 transistor. Reproduced with permission from Ref. 54.
    Basic principles of layer-by-layer processing sensors. (a) Conventional computing architectures. (b) Near-sensor computing architecture. (c) In-sensor computing architecture. Reproduced with permission from Ref. 63.
    Fig. 7. Basic principles of layer-by-layer processing sensors. (a) Conventional computing architectures. (b) Near-sensor computing architecture. (c) In-sensor computing architecture. Reproduced with permission from Ref. 63.
    Basic principles of sparse coding sensors. (a) Conventional linear response image sensing principle. (b) Image sensing and sparse coding principle.
    Fig. 8. Basic principles of sparse coding sensors. (a) Conventional linear response image sensing principle. (b) Image sensing and sparse coding principle.
    Basic principles of neural adaptation sensors. Reproduced with permission from Ref. 25.
    Fig. 9. Basic principles of neural adaptation sensors. Reproduced with permission from Ref. 25.
    FunctionsComponents of the human visual systemBio-inspired vision system
    ImagingEyeOptical lens
    Signal conversion and codingNeuromorphic image sensors
    Signal transmissionOptic nerveTransmission units
    Interpreting visual informationVisual centersANNs
    Table 1. Comparison of human and bio-inspired vision systems.
    Wenhao Tang, Qing Yang, Hang Xu, Yiyu Guo, Jiqiang Zhang, Chunfang Ouyang, Leixin Meng, Xu Liu. Review of bio-inspired image sensors for efficient machine vision[J]. Advanced Photonics, 2024, 6(2): 024001
    Download Citation