Efficient multi-bands image compression method for remote cameras

Jin Li; Fengdeng Liu; Zilong Liu

doi:10.3788/COL201715.022801

Abstract

In this Letter, we propose an efficient compression algorithm for multi-spectral images having a few bands. First, we propose a low-complexity removing spectral redundancy approach to improve compression performance. Then, a bit plane encoding approach is applied to each band to complete the compression. Finally, the experiments are performed on multi-spectral images. The experiment results show that the proposed compression algorithm has good compressive property. Compared with traditional approaches, the proposed method can decrease the average peak signal noise ratio by 0.36 dB at 0.5 bpp. The processing speed reaches 23.81 MPixels/s at the working frequency of 88 MHz, which is higher than the traditional methods. The proposed method satisfies the project application.

A class of multi-spectral CCD cameras is now heading for a high spatial resolution and multi-spectral bands. These cameras have a few bands, as many as ten bands. These cameras have larger amounts of data than panchromatic cameras. Therefore, it is necessary to compress multi-spectral images of these CCD cameras by using a higher performance compressor. However, these cameras use a so-called “mono-spectral” compressor. The compressor independently compresses each band, which is considered as a panchromatic image to complete. Since the redundancy between bands is not considered, the compression performance is lower. It is not suitable for multi-spectral cameras having a few bands. In this Letter, we provide an efficient compression algorithm for these cameras.

Considering the application for a satellite^[1,2], the complexity of multi-spectral compression approaches is not too high. For multi-spectral images, the compression approaches usually use prediction, transform, and vector quantization. The prediction-based methods use the previous encoded band to predict the current band. The prediction error is encoded by an entropy coding algorithm, such as an adaptive binary arithmetic encoding. The prediction-based approaches are widely used by 3D image (such as multi-spectral, hyper-spectral image) compression. For now, to cover 1D, 2D, and 3D coefficients, prediction algorithms include hundreds of predictors. For the on-board application, the main prediction methods were differential pulse-code modulation (DPCM), adaptive DPCM, Consultative Committee for Space Data Systems-Lossless Data Compression (CCSDS-LDC), CCSDSMultispectral and Hyperspectral Data Compression (CCSDS-MHDC), Joint Photographic Experts Group-Lossless Standard (JPEG-LS), and lookup table (LUT). They obtain the better compression performance for lossless compression. The prediction-based approaches are very simple and realized easily in hardware. However, the prediction-based methods have a much poorer performance on error resilience and a much lower lossy compression performance.

The transform-based multi-spectral compression methods mainly use the 3D transform. The 3D transform mainly includes two types: (1) an ordinary 3D transform, such as a 3D discrete wavelet transform (3D-DWT) or a 3D discrete cosine transform (3D-DCT); (2) a 2D transform in combination with other transform. For the first type of method, the ordinary 3D transform is applied to obtain transform coefficients. Then, the transformed coefficients are encoded by a 2D embedded zerotree wavelet (2D-EZW)^[3], 3D embedded block coding with optimized truncation (3D-EBCOT)^[4], 3D set partitioning in hierarchical trees (3D-SPIHT), a 3D set partitioning embedded block (3D-SPECK)^[5], and so on. The 3D transform-based methods can remove the spatial, spectral, and sign redundancy of multi-spectral images. Therefore, these methods have much better compression performance. However, they not only have the problem of complex storage managements but also have the high hardware complexity of compressors. In addition, these methods are only suitable for cameras having many bands. For the second type of method, the 2D transform is completed by a 2D-DWT, a 2D-DCT, a fast Fourier transform (FFT), a Walsh–Hadamard transform^[6], and so on. The other transform is completed by the Karhunen–Loeve Transform (KLT) or the principal component analysis (PCA)^[7,8]. The KLT can remove the spectral redundancy, and the 2D transform removes the spatial redundancy. These approaches also have much better compression performance and are also suitable for the multi-spectral images that have a few bands.

Sign up for Chinese Optics Letters TOC. Get the latest issue of Chinese Optics Letters delivered right to you！Sign up now

Considering the spectral redundancy for few bands multi-spectral images, we proposed a low-complexity compression algorithm based on removing spectral redundancy in combination with bit-plane encoding (RSRA-BPE). The proposed method has potential applications in an on-orbit remote sensing off-axis three-mirror camera^[9,10].

A multi-spectral CCD is composed of several CCD arrays in parallel and produces several bands simultaneously. Figure 1 shows the process of multi-spectral CCD imaging. The optics reflected and radiated by the ground target converges on the optical thin film of the CCD surface through the optical system. Each band CCD array captures optical energy to obtain the corresponding spectral band image. Each band image contains 1D spatial information of ground objects. At this point, the 1D spectral and 1D spatial image is obtained by this multi-spectral CCD camera. When the camera moves along the push-broom direction, other 1D spatial information of ground objects is obtained. Therefore, the multi-spectral CCD camera produces 3D images. Because several bands, including the same ground objects, are obtained simultaneously by the same multi-spectral CCD, the 3D images produced have spatial and spectral redundancy. For the same two spatial locations, image blocks A and B located separately in the two adjacent bands, the spectral correlation is defined as $ρ (A, B) = \frac{\sum_{i = 1}^{m} [(a_{i} - E [a]) (b_{i} - E [b])]}{\sqrt{\sum_{i = 1}^{m} {(a_{i} - E [a])}^{2} \sum_{i = 1}^{m} {(b_{i} - E [b])}^{2}}},$ (1)where $a_{i}$ is denoted as the pixels of A, $b_{i}$ is denoted as the pixels of B, $E [a]$ is denoted as the mean value of A, $E [b]$ is denoted as the mean value of B, and $m$ is denoted as the total number of pixels of one image block. According to Eq. (1), we test the spectral correlation of multi-spectral images having a few bands. We use the multi-spectral images that have four bands, which were taken by the SPOT satellite, and the ground standard resolution figure that was obtained by testing the multi-spectral time delay and integration CCD (TDICCD) camera in the calibration laboratory. The spectral correlation coefficients tested are shown in Fig. 2. The correlation coefficient $ρ$ is greater than 0.7 between the adjacent bands. Therefore, comparing the difference between many and few bands multi-spectral images, the multi-spectral images have a stronger spectral correlation, which is considered in the process of compression.

Figure 1.Process of multi-spectral CCD imaging.

Figure 2.Spectral correlation testing of multi-spectral images.

To weigh the computational complexity and compression performance, we proposed an efficient low-complexity removing spectral redundancy (LRSR) algorithm for multi-band CCD images.

We consider every two bands as one group. We define the total number of bands of the multi-spectral CCD camera as $P$ . So, all bands are divided into $P / 2$ groups. We use the Pearson-based approach to regroup the spectral bands. The correlation coefficients of two bands (denoted as $X$ and $Y$ ) can be expressed as^[11] $ρ_{X, Y} = \frac{Cov (X, Y)}{SD (X) SD (Y)} = \frac{E [(X - E (X)) (Y - E (Y))]}{\sqrt{E ({(X - E (X))}^{2})} \sqrt{E ({(Y - E (Y))}^{2})}} .$ (2) $ρ_{X, Y} \in [- 1, 1]$ if $ρ_{X, Y} > 0$ , so $X$ and $Y$ are relevant. If $ρ_{X, Y} = 0$ , then $X$ and $Y$ are irrelevant. If $ρ_{X, Y} < 0$ , then $X$ and $Y$ are inversely relevant. The two bands having the maximum value for $| ρ_{X, Y} |$ are considered as one group.

Each group performs removing spectral redundancy (RSR) to produce one main band and one sparse band. The energy of one band of the group is focused into the main band. The correlation of the group is removed. After all groups are processed, the next level RSR is performed. In the next level, the main bands are regrouped to be the first level. That is, every two main bands are placed in one group. In the level, each new group performs the RSR process. After all groups in the level are processed by RSR, the next level RSR is performed the same way as it was previously. Each level uses the same way to process. The number of the level denoted as $L$ is equal to $\log 2 (P)$ . When processing $level 1 = L$ , all bands are processed by RSR. The number of groups processed by RSR is denoted as $G$ . The $G$ can be expressed as $G = \frac{P}{2} + \frac{P}{4} + \dots + \frac{P}{2^{L}} .$ (3)

Figure 3 shows the process of RSR when the band’s number is 4. The process level number is 2. In the first level, there are two groups. Group 1 is processed by RSR to produce the main band $G_{1}$ and the sparse band $G_{1}^{'}$ . Group 2 is processed by RSR to produce the main band $G_{2}$ and the sparse band $G_{2}^{'}$ . In the second level, $G_{1}$ and $G_{2}$ are considered to be Group 3. Group 3 is processed by RSR to be the main band $G_{3}$ and the sparse band $G_{3}^{'}$ . Finally, all bands are processed to be one main band $G_{3}$ and three sparse bands $G_{3}^{'}$ , $G_{2}^{'}$ , and $G_{1}^{'}$ .

Figure 3.Process of RSR. The spectral bands number is 4.

The RSR is used to remove the correlation of two spectral bands in each group. Consider each band of multi-spectral images in a group as a matrix, the $i$ th band in a group is defined as $H_{i} = {[\begin{matrix} h_{1, 1, i} & h_{2,1, i} & h_{3, 1, i} & \dots & h_{L, 1, i} \\ h_{1, 2, i} & h_{2, 2, i} & h_{3, 2, i} & \dots & h_{L, 2, i} \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋮ \\ h_{1, M, i} & h_{2, M, i} & h_{3, M, i} & \dots & h_{L, M, i} \end{matrix}]}_{L \times M}, i = 1, 2,$ (4)where $L$ is line number of the band, and $M$ is the column number of the band. $H_{i}$ is composed of $M$ line vectors. Each line vector has $L$ elements. Each matrix is regrouped into a new matrix having only one line vector. The new line vector is organized in a line vector stack. Then, $H_{1}$ and $H_{2}$ are merged into a new matrix $H$ , which can be expressed as $H = [\begin{matrix} H_{1}^{'} \\ H_{2}^{'} \end{matrix}] = [\begin{matrix} \begin{matrix} h_{1, 1, 1} & h_{2, 1, 1} & h_{3, 1, 1} & \begin{matrix} \dots & h_{L, M, 1} \end{matrix} \end{matrix} \\ \begin{matrix} h_{1, 2, 2} & h_{2, 2, 2} & h_{3, 2, 2} & \begin{matrix} \dots & h_{L, M, 2} \end{matrix} \end{matrix} \end{matrix}],$ (5)where $H_{1}^{'}$ and $H_{2}^{'}$ are the row matrixes stacked by each row of $H_{1}$ and $H_{2}$ , respectively. The mean value of each band is denoted as $B_m$ , which can be expressed as $B_m = [\begin{matrix} {mean}_{1}, & {mean}_{2} \end{matrix}],$ (6)where ${mean}_{1}$ and ${mean}_{2}$ are the mean of $H_{1}$ and $H_{2}$ , respectively. By subtracting $B_m$ from the value of each band, the $M_sub$ can be expressed as $H = [\begin{matrix} \begin{matrix} h_{1, 1, 1} - {mean}_{1} & h_{2, 1, 1} - {mean}_{1} & h_{3, 1, 1} - {mean}_{1} & \begin{matrix} \dots & h_{L, M, 1} - {mean}_{1} \end{matrix} \end{matrix} \\ \begin{matrix} h_{1, 2, 2} - {mean}_{2} & h_{2, 2, 2} - {mean}_{2} & h_{3, 2, 2} - {mean}_{2} & \begin{matrix} \dots & h_{L, M, 2} - {mean}_{2} \end{matrix} \end{matrix} \end{matrix}] .$ (7)The covariance matrix of $H$ is denoted as $Cov (H)$ , which can be expressed as $Cov (H) = \frac{1}{4} H^{T} H = [\begin{matrix} {cov}_{11} & {cov}_{12} \\ {cov}_{21} & {cov}_{22} \end{matrix}] .$ (8)The eigenvectors of $Cov (H)$ are defined as $V = [\begin{matrix} v_{11} & v_{12} \\ v_{21} & v_{22} \end{matrix}] .$ (9)The eigenvector can be computed using the covariance matrix $v_{11} = v_{22} = \sqrt{\frac{1}{2} + \frac{({cov}_{11} - {cov}_{22})}{2 η}} = \sqrt{1 - v_{21}^{2}},$ (10) $v_{21} = - v_{12} = \frac{{cov}_{12}}{| {cov}_{12} |} \sqrt{\frac{1}{2} - \frac{{cov}_{11} - {cov}_{22}}{2 η}},$ (11) $η = \sqrt{{({cov}_{11} - {cov}_{22})}^{2} + 4 {cov}_{12} {cov}_{21}} .$ (12)The diagonal matrix $λ$ of $Cov (H)$ is defined as $λ = [\begin{matrix} λ_{1} & 0 \\ 0 & λ_{2} \end{matrix}] .$ (13)The diagonal matrix $λ$ can be computed by Eqs. (7) and (12) as $λ_{1} = \frac{{cov}_{11} + {cov}_{22} + η}{2} λ_{2} = \frac{{cov}_{11} + {cov}_{22} - η}{2},$ (14)since $Cov (H) V = V λ, Cov (H) = V λ V^{- 1}, V^{T} Cov (H) V = V^{T} V λ V^{- 1} V = V^{T} V λ .$ (15)In addition, $V^{T} V = [\begin{matrix} v_{11}^{2} + v_{22}^{2} & v_{11} v_{21} + v_{12} v_{22} \\ v_{11} v_{21} + v_{12} v_{22} & v_{21}^{2} + v_{22}^{2} \end{matrix}] .$ (16)Combined with Eqs. (9), (10), Eq. (15) can be expressed as $V^{T} V = [\begin{matrix} v_{11}^{2} + v_{22}^{2} & 0 \\ 0 & v_{21}^{2} + v_{22}^{2} \end{matrix}] .$ (17)So, Eq. (15) can be expressed as $V^{T} Cov (H) V = [\begin{matrix} λ_{1} (v_{11}^{2} + v_{22}^{2}) & 0 \\ 0 & λ_{2} (v_{21}^{2} + v_{22}^{2}) \end{matrix}] = Λ,$ (18)where $Λ$ is a diagonal matrix. In addition, there is $V^{T} Cov (H) V = \frac{1}{4} V^{T} H H^{T} V .$ (19)Consider that $G = V^{T} H$ , so $V^{T} Cov (H) V = \frac{1}{4} V^{T} H {(V^{T} H)}^{T} = \frac{1}{4} G G^{T} = Cov (G) .$ (20)According to Eqs. (14) and (16), there is $Cov (G) = Λ$ . Because $Cov (G)$ is a diagonal matrix, the value of the off-diagonal elements is zero. Therefore, the elements of $G$ are irrelevant. So, the removing correlation computation equation for multi-spectral images can be expressed as $G = V^{T} H .$ (21) $H$ can be obtained by multi-spectral images. According to Eq. (20), the spectral redundancy can be removed. In reality, our idea of RSR is the same with the KLT. However, our algorithm uses only two bands to perform the computation. Therefore, our algorithm has low complexity.

In general, the pixel number of the multi-spectral CCD is relatively large, such as 4096 pixels for each CCD. They can cause the high-complexity for Eq. (20). We divided each group into several sub-blocks (See Fig. 4), and each sub-block is processed by RSR. In a group, each band is divided into several sub-blocks. The sub-blocks that have the same spatial location of two bands are regrouped into new 3D images, which can be processed by RSR.

Figure 4.Spatial blocking.

Note that the smaller $K_{2} \times K_{1}$ can impact the compression performance. We test four multi-spectral images, and each group is $512 \times 512$ and has four bands.

From Fig. 5, when $K_{1} = K_{2} = 64$ , the compression performance begins to decrease. We use the other multi-spectral images that have a few bands to analyze the relationship between the peak signal-to-noise ratio (PSNR) and the size of the sub-block. The same result is obtained. We weigh the computation complexity against the compression performance, and consider the CCD output line by line. Therefore, we set $K_{2} = 64$ , and $K_{1} = N$ , where $N$ is the pixel number of each band.

Figure 5.Relation between compression performance and the size of the block, where (a) is the testing of the multi-spectral image, and (b) is the tested results.

Based on the LRSR algorithm, Fig. 6 shows the whole construction of our algorithm of multi-spectral images. The compression algorithm contains two parts: (1) the LRSR unit and (2) the removing spatial correlation (RSC) unit. The LRSR unit is used to remove spectral redundancy. The RSC unit is used to remove spatial redundancy. The LRSR unit has five stages: (1) grouping, (2) blocking, (3) 1-level RSR, (4) grouping, and (5) 2-level RSR. In each level of RSR, Eqs. (4)–(21) are calculated to remove spectral redundancy. The RSC unit has two stages: (1) spatial sparse and (2) bit-plane coding. In the spatial sparse stage, a 2D-DWT is applied to each band. The BPE of the CCSDS-Image Data Compression (IDC) is used to encode wavelet coefficients^[12].

Figure 6.Whole construct of the compression algorithm for multi-spectral images.

In order to evaluate the compression performance of the proposed RSRA-BPE algorithm, we use the self-development testing device. Figure 7 shows the testing experiment scheme. The testing platform includes an image simulation source, a multi-spectral compression system, ground camera test device, a compression server, and a display system. The compression server can produce the simulated multi-spectral images, which are transmitted to the image simulation source unit. The image simulation source unit adjusts the output line frequency, image size, and output time to simulate the multi-spectral CCD output. The multi-spectral compression system compresses the received simulated multi-spectral images. The compression system uses Virtex-PRO Xilinx FPGA with a 32 bit MicroBlaze processor. The compressed streams are received and decoded by the ground camera test device unit. The reconstructed image is transmitted to the compression server and display system.

Figure 7.Testing experiment scheme.

The compression server injects the SPOT multi-spectral images into the image simulation sources to test the compression performance of the proposed approach. Each group of multi-spectral images is $512 pixels \times 512 pixels \times 4$ . The depth of the pixels is 8 bits/pixel (bpp). We compare our algorithm with the CCSDS-IDC mono-spectral compressor. Figure 8 shows the tested PSNR of the two methods at 0.5–3 bpp. From Fig. 8, the PSNR of our algorithm improves to 0.36 dB more than the CCSDS-IDC mono-spectral compressor at 0.5 bpp. Because we use the multi-level RSR technology to remove the spectral redundancy, our method outperforms the CCSDS-IDC mono-spectral compression method.

Figure 8.Compression performance comparison with CCSDS-IDC.

We use multiple QuickBird satellite images to further compare the performance of our method with those of the CCSDS-IDC method and the Hadamard post-transform (H-PT) method. The compression server injects the QuickBird satellite testing multi-spectral images with four bands. The reconstructed images perform the PSNR analysis. We used different images from the testing image database to measure the corresponding PSNRs. The average PSNR is considered as the PSNR of the method. The calculated PSNRs of the different methods are shown in Table 1. We perform other image quality assessments by using the mean measure of structural similarity (MSSIM). The MSSIMs are based on the hypothesis that the human visual system (HVS) is highly adapted for extracting the structure information. The MSSIM values at different compressed ratios are shown in Fig. 9. Because we use several key technologies, such as the multi-level RSR to remove the spectral redundancy and the BPE method, our method outperforms the traditional on-orbit compression methods.

Figure 9.MSSIM values at different compression ratios.

Bit rate (bpp)	CCSDS-IDC (dB)	H-PT (dB)	Our method (dB)
0.5	48.8146	49.0258	49.7822
1.0	52.5619	53.0898	53.6402
1.5	55.3620	56.0288	56.5707
2.0	57.2189	57.8321	58.3466
2.5	58.7636	58.8558	59.1888
3.0	59.6379	59.6850	59.7778

Table 1. PSNR of Three Different Methods

View all Tables

In order to analyze the process speed of our algorithm, our algorithm is implemented by an FPGA processor. We use a self-developed CCD camera to test the compression time of our method. The line frequency of the CCD is 1.8094 kHz. The following compression speed of our algorithm is only used to perform the evaluations of compression speed. The compression algorithm is not optimized for the FPGA implementation. These evaluations are based on the lossy compression of remote sensing multi-spectral images with four bands. The size of each band is $3072 \times 128$ . Table 2 shows the comparison results of the processing speed of our algorithm with traditional approaches. From Table 1, the data throughput of our algorithm reaches 23.81 MPixels/s at an 88 MHz working frequency, which indicates that less time is spent than the JPEG2000, KLT, and 3D-SPIHT approaches. The compression time of $128 \times 3072$ needs only 16.51 ms. According to the different principles of CCD imaging, our compression algorithm can be optimized on an FPGA. An optimized implementation on an FPGA can spend minimal time. Overall, our algorithm has low complexity and high performance and is very suitable for space application.

Methods	Data Throughput (MSPS)^a
KLT^[13]	9.77
3D-SPIHT^[14]	16.04
JPEG2000^[15]	5.52
Our approach	23.81

Table 2. Data Throughput Comparison with Traditional Methods

View all Tables

In conclusion, we propose an efficient compression algorithm for multi-spectral images that have a few bands. First, we propose a low-complexity RSR approach to improve compression performance. Then, a BPE approach is applied to each band to complete compression. Finally, the experiments are performed on multi-spectral images. The experiment results show that the proposed compression algorithm has good compressive properties. Compared with traditional approaches, the proposed method can decrease the average PSNR by 0.36 dB at 0.5 bpp. However, the processing speed reaches 23.81 MPixels/s at the working frequency of 88 MHz, which is higher than traditional methods. The proposed the method satisfies the project application. Our method adopts the BPE method for encoding the transformed coefficients. However, BPE cannot remove the residual spectral redundancy. In the future, the distributed source coding method can replace BPE for integrating the proposed method for removing the residual spectral redundancy. The proposed method can also be integrated into a compressed sensing approach^[16] to reduce the computational complexity of the camera compressor.

References

[1] J. Li, Z. Liu. Appl. Opt., 55, 8070(2016).

[2] B. Lu, F. Wei, Z. Zhang, D. Xu, Z. Pan, D. Chen, H. Cai. Chin. Opt. Lett., 9, 091402(2015).

[3] K. Cheng, J. Dill. IEEE Trans. Geosci. Remote Sens., 52, 5765(2014).

[4] V. Sanchez. IEEE Trans. Biomed. Eng., 60, 397(2013).

[5] A. Karami. Int. Soc. Opt. Photonics, 9643, 96431J(2015).

[6] L. Wang, S. Zhao. Photon. Res., 4, 240(2016).

[7] J. Zabalza, J. Ren, J. Ren, Z. Liu, S. Marshall. Appl. Opt., 53, 4440(2014).

[8] L. Zhang, D. Liang, B. Li, Y. Kang, Z. Pan, D. Zhang, X. Ma. Photon. Res., 4, 115(2016).

[9] T. Yang, J. Zhu, G. Jin. Chin. Opt. Lett., 14, 060801(2016).

[10] Q. Cheng. Chin. Opt. Lett., 13, S11003(2015).

[11] K. Pearson. Trans. R. Soc. Lond., 187, 253(1896).

[12] J. Li, F. Xing, T. Sun, Z. You. Sci. World J., 2014, 738735(2014).

[13] I. Blane, J. Bartrina-Rapesta. IEEE Trans. Geosci. Remote Sens., 48, 2854(2010).

[14] F. Khelifi, A. Bouridane, F. Kurugollu. IEEE Trans. Multimedia, 10, 316(2008).

[15] J. González-Conejero, J. Bartrina-Rapesta, J. Serra-Sagristà. IEEE Geosci. Remote Sens. Lett., 7, 251(2010).

[16] Y. Li, Q. Li, J. Hu, Y. Zhao. Chin. Opt. Lett., 13, S11101(2015).