To eliminate the stripes of Remote Sensing Image (RSI), which is easy to cause blur or loss of detail, the direction and structure characteristics of clean image and stripes are analyzed and studied, a stripe separation method for RSI is proposed.For the stripes, the sparse structure feature is characterized by the group sparsity and smoothness of the stripes, the characteristic constraints are realized by combining the L2, 1 norm of stripe and L0 norm of the gradient along the direction of stripe.For clean image, the edge continuity is constrained by L1 norm.Considering the non-convexity of the constraint model, the algorithm based on Proximal Alternating Direction Method of Multipliers (PADMM) is used to solve the constraint model fast.Experimental results of simulation data and actual data show that the proposed algorithm can effectively remove the stripes and preserve the details of the original image efficiently, which fully verifies the feasibility and reliability of the algorithm.