• Chinese Journal of Quantum Electronics
  • Vol. 39, Issue 6, 899 (2022)
Jianming CHEN1、2、*, Xiangjin ZENG1、2, Liyun ZHONG1、2, Jianglei DI1、2, and Yuwen QIN1、2
Author Affiliations
  • 1[in Chinese]
  • 2[in Chinese]
  • show less
    DOI: 10.3969/j.issn.1007-5461.2022.06.006 Cite this Article
    CHEN Jianming, ZENG Xiangjin, ZHONG Liyun, DI Jianglei, QIN Yuwen. Research progress of image registration methods based on deep learning[J]. Chinese Journal of Quantum Electronics, 2022, 39(6): 899 Copy Citation Text show less
    References

    [1] Zitová B, Flusser J. Image registration methods: A survey[J]. Image and Vision Computing, 2003, 21(11): 977-1000.

    [2] Mo X Y, Yang F, Yin M X, et al. Survey on deep learning in medical image registration[J]. Journal of Chinese Computer Systems, 2021, 42(8): 1706-1714.

    [3] Lester H, Arridge S R. A survey of hierarchical non-linear medical image registration[J]. Pattern Recognition, 1999, 32(1): 129-149.

    [4] Goshtasby A. Registration of images with geometric distortions[J]. IEEE Transactions on Geoscience and Remote Sensing, 1988, 26(1): 60-64.

    [5] Friston K J, Ashburner J, Frith C D, et al. Spatial registration and normalization of images[J]. Human Brain Mapping, 1995, 3(3): 165-189.

    [6] Wells W M III, Viola P, Atsumi H, et al. Multi-modal volume registration by maximization of mutual information[J]. Medical Image Analysis, 1996, 1(1): 35-51.

    [7] Studholme C, Drapaca C, Iordanova B, et al. Deformation-based mapping of volume change from serial brain MRI in the presence of local tissue contrast change[J]. IEEE Transactions on Medical Imaging, 2006, 25(5): 626-639.

    [8] Balntas V, Johns E, Tang L L, et al. PN-Net: conjoined triple deep network for learning local image descriptors[J]. Computer Science, 2016, arXiv: preprints 1601.05030.

    [9] Kuglin C D, Hines D C. The phase correlation image registration method[C]. Proceedings of IEEE International Conference on Cybernetics and Society, 1975: 163-165.

    [10] Reddy B S, Chatterji B N. An FFT-based technique for translation, rotation, and scale-invariant image registration[J]. IEEE Transactions on Image Processing, 1996, 5(8): 1266-1271.

    [11] Rocco I, Arandjelovic R, Sivic J. Convolutional neural network architecture for geometric matching[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2017: 39-48.

    [12] Chauhan R, Ghanshala K K, Joshi R C. Convolutional neural network (CNN) for image detection and recognition[C]. First International Conference on Secure Cyber Computing and Communication (ICSCCC), 2018: 278-282.

    [13] Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.

    [14] Liu C, Shang Z W, Qin A Y. A multiscale image denoising algorithm based on dilated residual convolution network[C]. Image and Graphics Technologies and Applications, 2019: 193-203.

    [15] Ren S Q, He K M, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.

    [16] LeCun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code recognition[J]. Neural Computation, 1989, 1(4): 541-551.

    [17] LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.

    [18] Gu J X, Wang Z H, Kuen J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354-377.

    [19] Krizhevsky A, Sutskever I, Hinton G E. Image Net classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.

    [20] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[OL]. 2014, arXiv: 1409.1556, https://arxiv.org/abs/1409.1556.

    [21] Szegedy C, Liu W, Jia Y Q, et al. Going deeper with convolutions[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2015: 1-9.

    [22] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.

    [23] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation[C]. Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015, 2015: 234-241.

    [24] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144.

    [25] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]. Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010.

    [26] Wu G R, Kim M, Wang Q, et al. Unsupervised deep feature learning for deformable registration of MR brain images[C]. Medical Image Computing and Computer-Assisted Intervention, 2013, 16(Pt 2): 649-656.

    [27] Cheng X, Zhang L, Zheng Y F. Deep similarity learning for multimodal medical images[J]. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 2018, 6(3): 248-252.

    [28] Sedghi A, O’Donnell L J, Kapur T, et al. Image registration: Maximum likelihood, minimum entropy and deep learning[J]. Medical Image Analysis, 2021, 69: 101939.

    [29] Simonovsky M, Gutiérrez-Becker B, Mateus D, et al. A deep metric for multimodal registration[C]. Medical Image Computing and Computer-Assisted Intervention-MICCAI, 2016: 10-18.

    [30] Czolbe S, Krause O, Feragen A. DeepSim: Semantic similarity metrics for learned image registration[OL]. 2020, arXiv: 2011.05735, https://arxiv.org/abs/2011.05735.

    [31] Sloan J M, Goatman K A, Siebert J P. Learning rigid image registration—utilizing convolutional neural networks for medical image registration[C]. Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies, 2018.

    [32] Miao S, Wang Z J, Liao R. A CNN regression approach for real-time 2D/3D registration[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1352-1363.

    [33] Miao S, Wang Z J, Zheng Y F, et al. Real-time 2D/3D registration via CNN regression[C]. IEEE 13th International Symposium on Biomedical Imaging, 2016: 1430-1434.

    [34] Mohseni Salehi S S, Khan S, Erdogmus D, et al. Real-time deep pose estimation with geodesic loss for image-to-template rigid registration[J]. IEEE Transactions on Medical Imaging, 2019, 38(2): 470-481.

    [35] Zou M Y, Hu J R, Zhang H, et al. Rigid medical image registration using learning-based interest points and features[J]. Computers, Materials and Continua, 2019, 58(2): 511-525.

    [36] Chee E, Wu Z Z. AIRNet: Self-supervised affine registration for 3D medical images using neural networks[OL]. 2018, arXiv: 1810.02583, https://arxiv.org/abs/1810.02583.

    [37] Zheng J N, Miao S, Wang Z J, et al. Pairwise domain adaptation module for CNN-based 2-D/3-D registration[J]. Journal of Medical Imaging, 2018, 5(2): 021204.

    [38] Guo H T, Kruger M, Xu S, et al. Deep adaptive registration of multi-modal prostate images[J]. Computerized Medical Imaging and Graphics, 2020, 84: 101769.

    [39] Yang X, Kwitt R, Niethammer M. Fast predictive image registration[OL]. 2016, arXiv: 1607.02504, https://arxiv.org/abs/1607.02504.

    [40] Cao X H, Yang J H, Zhang J, et al. Deformable image registration based on similarity-steered CNN regression[C]. Medical Image Computing and Computer-Assisted Intervention, 2017: 300-308.

    [41] Cao X H, Yang J H, Zhang J, et al. Deformable image registration using a cue-aware deep regression network[J]. IEEE Transactions on Biomedical Engineering, 2018, 65(9): 1900-1911.

    [42] Teng X Z, Chen Y X, Zhang Y W, et al. Respiratory deformation registration in 4D-CT/cone beam CT using deep learning[J]. Quantitative Imaging in Medicine and Surgery, 2021, 11(2): 737-748.

    [43] Eppenhof K A J, Pluim J P W. Error estimation of deformable image registration of pulmonary CT scans using convolutional neural networks[J]. Journal of Medical Imaging, 2018, 5(2): 024003.

    [44] Sokooti H, de Vos B, Berendsen F. Nonrigid image registration using multi-scale 3D convolutional neural networks[C]. Medical Image Computing and Computer Assisted Intervention, 2017: 232-239.

    [45] Sokooti H, de Vos B, Berendsen F, et al. 3D convolutional neural networks image registration based on efficient supervised learning from artificial deformations[OL]. 2019, arXiv: 1908.10235, https://arxiv.org/abs/1908.10235.

    [46] Fu Y B, Lei Y, Wang T H, et al. Biomechanically constrained non-rigid MR-TRUS prostate registration using deep learning based 3D point cloud matching[J]. Medical Image Analysis, 2021, 67: 101845.

    [47] Hu Y P, Modat M, Gibson E, et al. Label-driven weakly-supervised learning for multimodal deformable image registration[C]. IEEE 15th International Symposium on Biomedical Imaging, 2018: 1070-1074.

    [48] Blendowski M, Hansen L, Heinrich M P. Weakly-supervised learning of multi-modal features for regularised iterative descent in 3D image registration[J]. Medical Image Analysis, 2021, 67(8): 101822.

    [49] Hering A, Kuckertz S, Heldmann S. Enhancing label-driven deep deformable image registration with local distance metrics for state-of-the-art cardiac motion tracking[C]. Bildverarbeitung für die Medizin, 2019: 309-314.

    [50] Shao W, Bhattacharya I, Soerensen S J C, et al. Weakly supervised registration of prostate MRI and histopathology images[C]. Medical Image Computing and Computer Assisted Intervention-MICCAI 2021, 2021: 98-107.

    [51] Zhu Q Y, Sun Y H, Wu Y, et al. Whole-brain functional MRI registration based on a semi-supervised deep learning model[J]. Medical Physics, 2021, 48(6): 2847-2858.

    [52] Zhu Z Y, Cao Y Q, Qin C C, et al. Joint affine and deformable three-dimensional networks for brain MRI registration[J]. Medical Physics, 2021, 48(3): 1182-1196.

    [53] Wang Y Q, Zhang J K, Cavichini M, et al. Robust content-adaptive global registration for multimodal retinal images using weakly supervised deep-learning framework[J]. IEEE Transactions on Image Processing, 2021, 30: 3167-3178.

    [54] Peng Y Z, Chen X J, Xiang D H, et al. Keypoint matching networks for longitudinal fundus image affine registration[C]. Proceedings of SPIE, 2021: 115963D.

    [55] Wang X Y, Mao L Z, Huang X J, et al. Multimodal MR image registration using weakly supervised constrained affine network[J]. Journal of Modern Optics, 2021, 68: 679-688.

    [56] Lei Y, Fu Y B, Wang T H, et al. 4D-CT deformable image registration using multiscale unsupervised deep learning[J]. Physics in Medicine and Biology, 2020, 65(8): 085003.

    [57] Hu Y P, Gibson E, Ghavami N, et al. Adversarial deformation regularization for training image registration neural networks[C]. Medical Image Computing and Computer Assisted Intervention, 2018: 774-782.

    [58] Chen X, Xia Y, Ravikumar N, et al. A deep discontinuity-preserving image registration network[C]. Medical Image Computing and Computer Assisted Intervention, 2021: 46-55.

    [59] Ma L F, Luo F, Yan J P, et al. Deep-learning based medical image registration pathway: Towards unsupervised learning[J]. Journal of Image and Graphics, 2021, 26(9): 2037-2057.

    [60] Fan J F, Cao X H, Yap P T, et al. BIRNet: Brain image registration using dual-supervised fully convolutional networks[J]. Medical Image Analysis, 2019, 54: 193-206.

    [61] Cao X H, Yang J H, Wang L, et al. Deep learning based inter-modality image registration supervised by intra-modality similarity[C]. Machine Learning in Medical Imaging, 2018: 55-63.

    [62] Yan P K, Xu S, Rastinehad A R, et al. Adversarial image registration with application for MR and TRUS image fusion[C]. Machine Learning in Medical Imaging, 2018: 197-204.

    [63] Arjovsky M, Chintala S, Bottou L. Wasserstein generative adversarial networks[C]. Proceedings of the 34th International Conference on Machine Learning, 2017: 214-223.

    [64] Qiu L, Ren H L. RSegNet: A joint learning framework for deformable registration and segmentation[J]. IEEE Transactions on Automation Science and Engineering, 2022, 19(3): 2499-2513.

    [65] Jaderberg M, Simonyan K, Zisserman A, et al. Spatial transformer networks[C]. Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2, 2015: 2017-2025.

    [66] de Vos B D, Berendsen F F, Viergever M A, et al. End-to-end unsupervised deformable image registration with a convolutional neural network[OL]. 2017, arXiv: 1704.06065, https://arxiv.org/abs/1704.06065.

    [67] de Vos B D, Berendsen F F, Viergever M A, et al. A deep learning framework for unsupervised affine and deformable image registration[J]. Medical Image Analysis, 2019, 52: 128-143.

    [68] Balakrishnan G, Zhao A, Sabuncu M R, et al. An unsupervised learning model for deformable medical image registration[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 9252-9260.

    [69] Avants B B, Epstein C L, Grossman M, et al. Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain[J]. Medical Image Analysis, 2008, 12(1): 26-41.

    [70] Balakrishnan G, Zhao A, Sabuncu M R, et al. VoxelMorph: A learning framework for deformable medical image registration[J]. IEEE Transactions on Medical Imaging, 2019, 38(8): 1788-1800.

    [71] Zhao S Y, Dong Y, Chang E, et al. Recursive cascaded networks for unsupervised medical image registration[C]. IEEE/CVF International Conference on Computer Vision (ICCV), 2019: 10599-10609.

    [72] Zhang L T, Zhou L, Li R Y, et al. Cascaded feature warping network for unsupervised medical image registration[C]. IEEE 18th International Symposium on Biomedical Imaging, 2021: 913-916.

    [73] Dalca A V, Balakrishnan G, Guttag J, et al. Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces[J]. Medical Image Analysis, 2019, 57: 226-236.

    [74] Dalca A V, Balakrishnan G, Guttag J, et al. Unsupervised learning for fast probabilistic diffeomorphic registration[C]. Medical Image Computing and Computer Assisted Intervention, 2018: 729-738.

    [75] Mok T C W, Chung A C S. Fast symmetric diffeomorphic image registration with convolutional neural networks[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 4643-4652.

    [76] Chen Q G, Li Z W, Lui L M. A learning framework for diffeomorphic image registration based on quasi-conformal geometry[J]. Computer Science, 2021, arXiv: preprints2110.10580.

    [77] Kim B, Kim D H, Park S H, et al. CycleMorph: Cycle consistent unsupervised deformable image registration[J]. Medical Image Analysis, 2021, 71: 102036.

    [78] Xu Z, Luo J, Yan J P, et al. F3RNet: Full-resolution residual registration network for deformable image registration[J]. International Journal of Computer Assisted Radiology and Surgery, 2021, 16(6): 923-932.

    [79] Han L Y, Dou H R, Huang Y Z, et al. Deformable Registration of Brain MR Images Via a Hybrid Loss[M]. Biomedical Image Registration, Domain Generalisation and Out-of-Distribution Analysis, Springer International Publishing, 2022: 141-146.

    [80] Kori A, Krishnamurthi G. Zero shot learning for multi-modal real time image registration[OL]. 2019, arXiv: 1908.06213, https://arxiv.org/abs/1908.06213.

    [81] Xu Z, Yan J P, Luo J, et al. Unsupervised multimodal image registration with adaptative gradient guidance[C]. International Conference on Acoustics, Speech, and Signal Processing, 2021: 1225-1229.

    [82] Sideri-Lampretsa V, Kaissis G, Rueckert D. Multi-modal unsupervised brain image registration using edge maps[C]. IEEE 19th International Symposium on Biomedical Imaging, 2022: 1-5.

    [83] Pluim J P W, Maintz J B A, Viergever M A. Image registration by maximization of combined mutual information and gradient information[C]. IEEE Transactions on Medical Imaging, 19(8): 809-814.

    [84] Tian Y T, Hu Y, Ma Y H, et al. Multi-scale U-net with edge guidance for multimodal retinal image deformable registration[C]. 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2020: 1360-1363.

    [85] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[OL]. Computer Science, 2020, arXiv: preprint 2010.11929.

    [86] Chen J Y, He Y F, Frey E C, et al. ViT-V-Net: Vision transformer for unsupervised volumetric medical image registration[OL]. Computer Science, 2021, arXiv:preprints 2104.06468.

    [87] Liu Z, Lin Y T, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C]. IEEE/CVF International Conference on Computer Vision (ICCV), 2021: 9992-10002.

    [88] Chen J, Frey E C, He Y, et al. TransMorph: transformer for unsupervised medical image registration[OL]. Computer Science, 2021, arXiv: preprints 2111.10480.

    [89] Wang Y B, Qian W, Zhang X M. A transformer-based network for deformable medical image registration[OL].2022, arXiv: 2202.12104, https://arxiv.org/abs/2202.12104.

    [90] Wang Z H, Delingette H. Attention for image registration (AiR): An unsupervised transformer approach[OL]. 2021, arXiv: 2105.02282, https://arxiv.org/abs/2105.02282.

    [91] Singh N K, Raza K. Medical Image Generation Using Generative Adversarial Networks: A Review[M]. Health Informatics: A Computational Perspective in Healthcare. Singapore: Springer Singapore, 2021: 77-96.

    [92] Zhang X Y, Jian W J, Chen Y, et al. Deform-GAN: An unsupervised learning model for deformable registration[OL]. 2020, arXiv: pre-prints 2002.11430.

    [93] Arar M, Ginger Y, Danon D, et al. Unsupervised multi-modal image registration via geometry preserving image-to-image translation[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 13407-13416.

    [94] Qin C, Shi B B, Liao R, et al. Unsupervised deformable registration for multi-modal images via disentangled representations[OL]. 2019, arXiv: pre-prints 1903.09331.

    [95] Xu Z, Luo J, Yan J P, et al. Adversarial uni- and multi-modal stream networks for multimodal image registration[C]. Medical Image Computing and Computer Assisted Intervention, 2020: 222-232.

    [96] Hoopes A, Hoffmann M, Fischl B, et al. HyperMorph: Amortized Hyperparameter Learning for Image Registration[C]. Information Processing in Medical Imaging, 2021: 3-17.

    [97] Mok T C W, Chung A C S. Conditional deformable image registration with convolutional neural network[OL]. 2021, arXiv: pre-prints 2106.12673.

    [98] Jia D Q, Gao S Q, Chen Q L, et al. A low-rank representation for unsupervised registration of medical images[OL]. 2021, arXiv: preprint 2105.09548.

    CHEN Jianming, ZENG Xiangjin, ZHONG Liyun, DI Jianglei, QIN Yuwen. Research progress of image registration methods based on deep learning[J]. Chinese Journal of Quantum Electronics, 2022, 39(6): 899
    Download Citation