• Photonics Research
  • Vol. 11, Issue 2, 299 (2023)
Guoqing Ma1、2, Junjie Yu1、2、3, Rongwei Zhu1、2, and Changhe Zhou1、2、*
Author Affiliations
  • 1Laboratory of Information Optics and Optoelectronic Technology, Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China
  • 2Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China
  • 3e-mail: Junjiey@siom.ac.cn
  • show less
    DOI: 10.1364/PRJ.472741 Cite this Article Set citation alerts
    Guoqing Ma, Junjie Yu, Rongwei Zhu, Changhe Zhou. Optical multi-imaging–casting accelerator for fully parallel universal convolution computing[J]. Photonics Research, 2023, 11(2): 299 Copy Citation Text show less
    References

    [1] Y. Lecun, L. Bottou, Y. Bengio, P. Haffner. Gradient-based learning applied to document recognition. Proc. IEEE, 86, 2278-2324(1998).

    [2] A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst., 25, 1097-1105(2013).

    [3] Y. LeCun, Y. Bengio, G. Hinton. Deep learning. Nature, 521, 436-444(2015).

    [4] J. Cong, B. Xiao. Minimizing computation in convolutional neural networks. International Conference on Artificial Neural Networks, 281-290(2014).

    [5] T. F. De Lima, H.-T. Peng, A. N. Tait, M. A. Nahmias, H. B. Miller, B. J. Shastri, P. R. Prucnal. Machine learning with neuromorphic photonics. J. Lightwave Technol., 37, 1515-1534(2019).

    [6] Y. Ito, R. Matsumiya, T. Endo. OOC-cuDNN: accommodating convolutional neural networks over GPU memory capacity. IEEE International Conference on Big Data, 183-192(2017).

    [7] K. He, X. Zhang, S. Ren, J. Sun. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition, 770-778(2016).

    [8] G. Wetzstein, A. Ozcan, S. Gigan, S. Fan, D. Englund, M. Soljačić, C. Denz, D. A. B. Miller, D. Psaltis. Inference in artificial intelligence with deep optics and photonics. Nature, 588, 39-47(2020).

    [9] B. J. Shastri, A. N. Tait, T. F. de Lima, W. H. P. Pernice, H. Bhaskaran, C. D. Wright, P. R. Prucnal. Photonics for artificial intelligence and neuromorphic computing. Nat. Photonics, 15, 102-114(2021).

    [10] P. Ambs. Optical computing: a 60-year adventure. Adv. Opt. Photon., 2010, 1-15(2010).

    [11] A. Maréchal, P. Croce. Un filtre de fréquences spatiales pour l’amélioration du contraste des images optiques. C. R. Acad. Sci., 237(1953).

    [12] L. De Marinis, M. Cococcioni, P. Castoldi, N. Andriolli. Photonic neural networks: a survey. IEEE Access, 7, 175827(2019).

    [13] P. R. Prucnal, B. J. Shastri. Neuromorphic Photonics(2017).

    [14] F. Thomas, B. J. Shastri, A. N. Tait, M. A. Nahmias, P. R. Prucnal. Progress in neuromorphic photonics. Nanophotonics, 6, 577-599(2017).

    [15] Q. Zhang, H. Yu, M. Barbiero, B. Wang, M. Gu. Artificial neural networks enabled by nanophotonics. Light Sci. Appl., 8, 42(2019).

    [16] Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, M. Soljačić. Deep learning with coherent nanophotonic circuits. Nat. Photonics, 11, 441-446(2017).

    [17] X. Xu, M. Tan, B. Corcoran, J. Wu, A. Boes, T. G. Nguyen, S. T. Chu, B. E. Little, D. G. Hicks, R. Morandotti, A. Mitchell, D. J. Moss. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature, 589, 44-51(2021).

    [18] H. Bagherian, S. Skirlo, Y. Shen, H. Meng, V. Ceperic, M. Soljacic. On-chip optical convolutional neural networks. arXiv(2018).

    [19] X. Lin, Y. Rivenson, N. T. Yardimci, M. Veli, Y. Luo, M. Jarrahi, A. Ozcan. All-optical machine learning using diffractive deep neural networks. Science, 361, 1004-1008(2018).

    [20] A. Silva, F. Monticone, G. Castaldi, V. Galdi, A. Alù, N. Engheta. Performing mathematical operations with metamaterials. Science, 343, 160-163(2014).

    [21] J. Feldmann, N. Youngblood, C. D. Wright, H. Bhaskaran, W. H. P. Pernice. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature, 569, 208-214(2019).

    [22] J. Feldmann, N. Youngblood, M. Karpov, H. Gehring, X. Li, M. Stappers, M. Le Gallo, X. Fu, A. Lukashchuk, A. S. Raja, J. Liu, C. D. Wright, A. Sebastian, T. J. Kippenberg, W. H. P. Pernice, H. Bhaskaran. Parallel convolutional processing using an integrated photonic tensor core. Nature, 589, 52-58(2021).

    [23] F. Ashtiani, A. J. Geers, F. Aflatouni. An on-chip photonic deep neural network for image classification. Nature, 606, 501-506(2022).

    [24] S. Xu, J. Wang, R. Wang, J. Chen, W. Zou. High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays. Opt. Express, 27, 19778-19787(2019).

    [25] H. Dammann, E. Klotz. Coherent optical generation and inspection of two-dimensional periodic structures. Opt. Acta, 24, 505-515(1977).

    [26] C. Zhou, L. Liu. Numerical study of Dammann array illuminators. Appl. Opt., 34, 5961-5969(1995).

    [27] J. Yu, C. Zhou, W. Jia, W. Cao, S. Wang, J. Ma, H. Cao. Three-dimensional Dammann array. Appl. Opt., 51, 1619-1630(2012).

    [28] J. Chang, V. Sitzmann, X. Dun, W. Heidrich, G. Wetzstein. Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification. Sci. Rep., 8, 12324(2018).

    [29] M. Miscuglio, Z. Hu, S. Li, J. K. Grorge, R. Capanna, H. Dalir, P. M. Bardet, P. Gupta, V. J. Sorger. Massively parallel amplitude-only Fourier neural network. Optica, 7, 1812-1819(2020).

    [30] C. Zhou, L. Liu, Z. Wang. Binary-encoded vector–matrix multiplication architecture. Opt. Lett., 17, 1800-1802(1992).

    [31] L. Liu, G. Li, Y. Yin. Optical complex matrix–vector multiplication with negative binary inner products. Opt. Lett., 19, 1759-1761(1994).

    [32] H. Qin, R. Gong, X. Liu, X. Bai, J. Song, N. Sebe. Binary neural networks: a survey. Pattern Recogn., 105, 107281(2020).

    [33] M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, Y. Bengio. Binarized neural networks training neural networks with weights and activations constrained to +1 or −1. arXiv(2016).

    [34] C. Zhou, J. Yu, G. Li, G. Ma. Roadmap of optical computing. Proc. SPIE, 11898, 118981B(2021).

    [35] https://www.nvidia.com/en-us/deep-learning-ai/products/titan-rtx/. https://www.nvidia.com/en-us/deep-learning-ai/products/titan-rtx/

    [36] P. Minzioni, C. Lacava, T. Tanabe, J. Dong, X. Hu, G. Csaba, W. Porod, G. Singh, A. E. Willner, A. Almaiman, V. Torres-Company, J. Schröder, A. C. Peacock, M. J. Strain, F. Parmigiani, G. Contestabile, D. Marpaung, Z. Liu, J. E. Bowers, L. Chang, S. Fabbri, M. R. Vázquez, V. Bharadwaj, S. M. Eaton, P. Lodahl, X. Zhang, B. J. Eggleton, W. J. Munro, K. Nemoto, O. Morin, J. Laurat, J. Nunn. Roadmap on all-optical processing. J. Opt., 21, 063001(2019).

    [37] J. Wang, J.-Y. Yang, I. M. Fazal, N. Ahmed, Y. Yan, H. Huang, Y. Ren, Y. Yue, S. Dolinar, M. Tur, A. E. Willner. Terabit free-space data transmission employing orbital angular momentum multiplexing. Nat. Photonics, 6, 488-496(2012).

    [38] https://www.top500.org/system/180047/. https://www.top500.org/system/180047/

    [39] T. Wang, S.-Y. Ma, L. G. Wright, T. Onodera, B. C. Richard, P. L. McMahon. An optical neural network using less than 1 photon per multiplication. Nat. Commun., 13, 123(2022).

    [40] S. Gupta, A. Agrawal, K. Gopalakrishnan, P. Narayanan. Deep learning with limited numerical precision. arXiv(2015).

    [41] M. A. Nahmias, T. F. de Lima, A. N. Tait, H.-T. Peng, B. J. Shastri, P. P. Prucnal. Photonic multiply-accumulate operations for neural networks. IEEE J. Quantum Electron., 26, 7701518(2020).

    [42] J. Han, A. Jentzen, E. Weinan. Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci. USA, 115, 8505-8510(2018).

    [43] L. Mennel, J. Symonowicz, S. Wachter, D. K. Polyushkin, A. J. Molina-Mendoza, T. Mueller. Ultrafast machine vision with 2D material neural network image sensors. Nature, 579, 62-66(2020).

    [44] T. Zhou, X. Lin, J. Wu, Y. Chen, H. Xie, Y. Li, J. Fan, H. Wu, L. Fang, Q. Dai. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photonics, 15, 367-373(2021).

    [45] T. Zhou, L. Fang, T. Yan, J. Wu, Y. Li, J. Fan, H. Wu, X. Lin, Q. Dai. In situ optical backpropagation training of diffractive optical neural networks. Photon. Res., 8, 940-953(2020).

    [46] M. Gruber. Multichip module with planar-integrated free-space optical vector-matrix-type interconnects. Appl. Opt., 43, 463-470(2004).

    [47] G. Mínguez-Vega, M. Gruber, J. Jahns, J. Lancis. Achromatic optical Fourier transformer with planar-integrated free-space optics. Appl. Opt., 44, 229-235(2005).

    [48] Y. Zhang, C. Fowler, J. Liang, B. Azhar, M. Y. Shalaginov, S. Deckoff-Jones, S. An, J. B. Chou, C. M. Roberts, V. Liberman, M. Kang, C. Ríos, K. A. Richardson, C. Rivero-Baleine, T. Gu, H. Zhang, J. Hu. Electrically reconfigurable non-volatile metasurface using low-loss optical phase-change material. Nat. Nanotechnol., 3, 661-666(2021).

    [49] Z. Wu, M. Zhou, E. Khoram, B. Liu, Z. Yu. Neuromorphic metasurface. Photon. Res., 8, 46-50(2020).

    [50] H. Kwon, D. Sounas, A. Cordaro, A. Polman, A. Alù. Nonlocal metasurfaces for optical signal processing. Phys. Rev. Lett., 121, 173004(2018).

    Guoqing Ma, Junjie Yu, Rongwei Zhu, Changhe Zhou. Optical multi-imaging–casting accelerator for fully parallel universal convolution computing[J]. Photonics Research, 2023, 11(2): 299
    Download Citation