There are rich edge and texture features in the remote sensing image. The common integer wavelet transform (IWT) can only reduce the image energy of horizontal and vertical edges or textures of high frequency subbands, while that of non-horizontal and vertical edges or textures are still high. For resolving the above problem, a new remote sensing image compression algorithm using adaptive directional lifting-based integer wavelet transform (ADL-IWT) and partial even optimal threshold (PEOT) is presented. The new algorithm can reduce the non-horizontal and vertical edge and texture energy of remote sensing image, by using ADL-IWT. Additionally, the new algorithm optimizes partial encoding quantization threshold intervals to improve the search efficiency of the significant coefficients on these threshold intervals. Experimental results based on the SPECK coding algorithm with ADL-IWT and PEOT show that the new algorithm can not only support both lossy and lossless compression of remote sensing image using a single bitstream, but also perform better in peak signal to noise ratio (PSNR) than the SPECK algorithm using the common IWT for lossy compression.