Since the compressed sensing video reconstruction frame by frame suffers interframe dithering, a multi-frame reconstruction algorithm of compressed sensing by group is presented. First, observe randomly frame by frame, and then make use of interframe correlation as three-dimensional signal. Second, in each step of the iterative process, update the reconstructed video by using the residuals. Finally, carry out the signal threshold processing based on three-dimensional transform, such as the 3D Dual-Tree Complex Wavelet Transform etc. Experimental results show that it can precisely reconstruct the original video by using randomly observed values, and achieve a higher signal to noise ratio. It is shown that algorithm uses the relevance of video frames effectively to eliminate the interframe dithering in reconstruction frame by frame.