SpectraTr: A novel deep learning model for qualitative analysis of drug spectroscopy based on transformer structure

[in Chinese]; [in Chinese]; [in Chinese]; [in Chinese]; [in Chinese]; [in Chinese]; [in Chinese]

doi:10.1142/s1793545822500213

[1] X. Miao, Y. Miao, H. Gong, S. Tao, Y. Chen, "NIR spectroscopy coupled with chemometric algorithms for the prediction of cadmium content in rice samples," Spectrochim. Acta A. Mol. Biomol. Spectrosc. 257, 119700 (2021).

[2] C. Yu, S. Quan, C. Yang, C. Zhang, H. Zang, "Determination of the immunoglobulin G precipitation end-point by an intelligent near-infrared spectroscopy system," J. Innov. Opt. Health Sci. 14(3), 2150007 (2021).

[3] L. Yun, C. Tf, B. Msa, "Quantitative NIR spectroscopy for determination of degree of polymerisation of historical paper," Chemometr. Intell. Lab. Syst. 214, 104337 (2021).

[4] W. Chen, H. Li, F. Zhang, W. Xiao, Y. Du, "Handheld short-wavelength NIR spectroscopy for rapid determination of sugars and carbohydrate in fresh juice with sampling error profile analysis," Infrared Phys. Technol. 115, 103732 (2021).

[5] H. Wang, R. Liu, L. Nie, D. Xu, H. Zang, "Spectra selection methods: A novel optimization way for treating dynamic spectra and in-line near infrared modeling," J. Innov. Opt. Health Sci. 13(4), 2050015 (2020).

[6] L. Haoxiang, Z. Jing, L. Lingqiao, L. Zhenbing, Y. Huihua, F. Yanchun, Y. Lihui, "Least angle regression combined with competitive adaptive re-weighted sampling for NIR spectral wavelength selection," Spectrosc. Spectral Anal. 41(6), 1782–1788 (2021).

[7] P. Mishra, J. M. Roger, F. Marini, A. Biancolillo, D. N. Rutledge, "Parallel pre-processing through orthogonalization (PORTO) and its application to near-infrared spectroscopy," Chemometr. Intell. Lab. Syst. 212, 104190 (2021).

[8] G. Borui, Y. Huihua, Z. Weidong, F. Yanchun, Y. Lihui, H. Changqin, "Stacked contractive autoencoders application in identification of pharmaceuticals," Spectrosc. Spectral Anal. 39(1), 96–102 (2019).

[9] M. P. Milali, S. S. Kiware, N. J. Govella, F. Okumu, N. Bansal, S. Bozdag et al., "An autoencoder and artificial neural network-based method to estimate parity status of wild mosquitoes from near-infrared spectra," PLoS One 15(6), e0234557 (2020).

[10] J. Acquarelli, T. van Laarhoven, J. Gerretzen, T. N. Tran, L. M. Buydens, E. Marchiori, "Convolutional neural networks for vibrational spectroscopic data analysis," Analy. Chim. Acta 954, 22–31 (2017).

[11] X. Zhang, T. Lin, J. Xu, X. Luo, Y. Ying, "DeepSpectra: An end-to-end deep learning approach for quantitative spectral analysis," Anal. Chim. Acta 1058, 48–57 (2019).

[12] P. S. Vasafi, O. Paquet-Durand, K. Brettschneider, J. Hinrichs, B. Hitzmann, "Anomaly detection during milk processing by autoencoder neural network based on near-infrared spectroscopy," J. Food Eng. 299, 110510 (2021).

[13] L. Lingqiao, P. Xipeng, F. Yanchun, Y. Lihui, Y. Huihua, "Deep convolution network application in identification of multi-variety and multi-manufacturer pharmaceutical," Spectrosc. Spectral Anal. 39(11), 3606–3613 (2019).

[14] S. Jo, W. Sohng, H. Lee, H. Chuang, "Evaluation of an autoencoder as a feature extraction tool for near-infrared spectroscopic discriminant analysis," Food Chem. 331, 127332 (2020).

[15] X. Zhang, J. Yang, T. Lin, Y. Ying, "Food and agroproduct quality evaluation based on spectroscopy and deep learning: A review," Trends Food Sci. Technol. 112, 431–441 (2021).

[16] Y. Chen, Z. Wang, "Feature selection based convolutional neural network pruning and its application in calibration modeling for NIR spectroscopy," Chemometr. Intell. Laboratory Syst. 191, 103–108 (2019).

[17] H. Qiao, X. Shi, H. Chen, J. Lyu, S. Hong, "Effective prediction of soil organic matter by deep SVD concatenation using FT-NIR spectroscopy," Soil Tillage Res. 215, 105223 (2022).

[18] A. Khan, A. Sohail, U. Zahoora, A. S. Qureshi, "A survey of the recent architectures of deep convolutional neural networks," Artif. Intell. Rev. 53(8), 5455–5516 (2020).

[19] D. Passos, P. Mishra, "An automated deep learning pipeline based on advanced optimisations for leveraging spectral classification modelling," Chemometr. Intell. Laboratory Syst. 215(1), 104354 (2021).

[20] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit et al., "Attention is all you need," Adv. Neural Inf. Process. Syst. 30(NIPS 2017), 5998–6008 (2017). DOI:10.48550/arXiv.1706.03762.

[21] S. Khan, M. Naseer, M. Hayat, S. W. Zamir et al., "Transformers in vision: A survey," arXiv:2101.01169 (2021).

[22] T. Lin, Y. Wang, X. Liu, X. Qiu, "A Survey of Transformers," arXiv:2106.04554 (2021).

[23] J. Wang, C. Y. Hsieh, M. Wang, X. Wang, Z. Wu et al., "Multi-constraint molecular generation based on conditional transformer, knowledge distillation and reinforcement learning," Nat. Mach. Intell. 3(10), 914–922 (2021).

[24] V. Venkatasubramanian, V. Mann, "Artificial intelligence in reaction prediction and chemical synthesis," Curr. Opin. Chem. Eng. 36, 100749 (2022).

[25] J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov et al., "Highly accurate protein structure prediction with AlphaFold," Nature 596(7873), 583–589 (2021).

[26] Y. Dai, Y. Gao, F. Liu, "Transmed: Transformers advance multi-modal medical image classification," Diagnostics 11(8), 1384 (2021).

[27] B. Yun, Y. Wang, J. Chen, H. Wang, W. Shen, "SpecTr: Spectral transformer for hyperspectral pathology image segmentation," arXiv:2103.03604 (2021).

[28] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, N. Houlsby, "An image is worth 16x16 words: Transformers for image recognition at scale," arXiv:2010.11929 (2020).

[29] L. Huang, S. Guo, Y. Wang, S. Wang, T. Bai, "Attention based residual network for medicinal fungi near infrared spectroscopy analysis," Math. Biosci. Eng. 16(4), 3003–3017 (2019).

[30] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, Proc. IEEE Conf.Computer Vision and Pattern Recognition, pp. 770–778 (IEEE, 2016).

[31] L. Lingqiao, L. Yanhui, Y. Linlin, Y. Huihua, F. Yanchun, Y. Lihui, H. Changqin, "Data augmentation of Raman spectral and its application research based on DCGAN," Spectrosc. Spectral Anal. 41(2), 400–407 (2021).