With the aim to improve the speed and reduce the time of color video stream compression transform, this paper proposed a fast Discrete Cosine Transform (DCT) algorithm based on multi-dimensional vector matrix. The algorithm combined a fast DCT based on multi-dimensional vector orthogonal matrix transform algorithm with a existing fast DCT algorithm for the first time. Firstly, it corrected and supplemented the fast 1D-DCT algorithm with strong practicality, and extended two most representative 2D-DCT fast algorithms, the rank decomposition method and the linear combination method, to the multi-dimensional field. Then, two DCT fast algorithms and their butterfly diagrams were deduced by combining multi-dimensional vector matrix theory, and the times of multiplications were analyzed. Finally, two methods were applied to test of the video in the standard video library meanwhile comparative experiments were performed. The experimental results show that the required time of the two methods is both less 0.25 s in DCT and these meet the real-time requirements of video. The proposed two methods ensure the reconstructed video quality, and not only reduce the computation time significantly, but also expand the DCT fast algorithm from a low-dimension to the multi-dimension.