The compressive sensing technology has a great potential in high-dimensional vision processing. The existing video reconstruction methods utilize the multihypothesis prediction to derive the residual sparse model from key frames. However, these methods cannot fully utilize the temporal correlation among multiple frames. Therefore, this paper proposes the video compressive sensing reconstruction via long-short-term double-pattern prediction, which consists of four main phases: the first phase reconstructs each frame independently; the second phase adaptively updates multiple reference frames; the third phase selects the hypothesis matching patches from current reference frames; the fourth phase obtains the reconstruction results by using the patches to build the residual sparse model. The experimental results demonstrate that as compared with the state-of-the-art methods, the proposed methods can obtain better prediction accuracy and reconstruction quality for video compressive sensing.