Liang Lin, Binbin Yang. From Perception to Creation: Exploring Frontier of Image and Video Generation Methods[J]. Acta Optica Sinica, 2023, 43(15): 1510002
- Acta Optica Sinica
- Vol. 43, Issue 15, 1510002 (2023)
Fig. 1. Overview of generative adversarial network (GAN) principle
Fig. 2. Overview of variational auto-encoder (VAE)
Fig. 3. Overview of flow-based generative model
Fig. 4. Overview of diffusion model
Fig. 5. Comparison of different generative models
Fig. 6. Overview of image and video generation models
Fig. 7. Class-conditioned image generation[95]
Fig. 8. Text-conditioned image generation[96]
Fig. 9. Text-to-image generation results of Stable Diffusion[97]
Fig. 10. Weight-tuning-based image customization[102]
Fig. 11. Token-learning-based image customization[103]
Fig. 12. Image customization with multi-concept composition[104]
Fig. 13. Mask-region-based text-to-image editing[105]
Fig. 14. Prompt-editing-based text-to-image editing[106]
Fig. 15. Embedding-interpolation-based text-to-image editing[107]
Fig. 16. Generated videos of Imagen Video[108]
Fig. 17. VIDM framework using two diffusion model to generate video content and action information respectively[111]
Fig. 18. PVDM framework that represents the video as three two-dimensional hidden variables, and thus uses the two-dimensional diffusion model for training[112]
Fig. 19. Text-to-video generation results of Make-A-Video[113]
Fig. 20. VideoFusion framework that uses pre-trained text-to-image diffusion model to generate base frame and uses video data to train a residual noise generator[114]
Fig. 21. Video editing based on input image or text prompt[115]
Fig. 22. One shot text-to-video generation[116]
|
Table 1. FID comparison of different text-to-image pre-trained models on MS-COCO dataset
|
Table 2. Performance comparison of different class-to-video generation methods on UCF-101 dataset and Sky Time-lapse dataset
Set citation alerts for the article
Please enter your email address
© Copyright 2018-2021 | Chinese Laser Press. All Rights Reserved 沪ICP备15018463号-20