• Chinese Optics Letters
  • Vol. 14, Issue 9, 091001 (2016)
Kejia Wang1、2, Ziliang Ping3、4, and Yunlong Sheng1、*
Author Affiliations
  • 1Center for Optics, Photonics and Lasers, Department of Physics, Physical Engineering and Optics, Laval University, Quebec G1V0A6, Canada
  • 2Inner Mongolia Electronic Information Vocational Technical College, Huhhot 010070, China
  • 3School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China
  • 4Inner Mongolia Normal University, Huhhot 010020, China
  • show less
    DOI: 10.3788/COL201614.091001 Cite this Article Set citation alerts
    Kejia Wang, Ziliang Ping, Yunlong Sheng. Development of image invariant moments—a short overview[J]. Chinese Optics Letters, 2016, 14(9): 091001 Copy Citation Text show less

    Abstract

    We give a brief overview on the more than 50 years of development of the moment-based image description, the moment invariants, and the orthogonal moments. Some basic ideas for significantly improving the performance of the image moment-based methods, such as the use of the low-order radial moments for reducing information suppression drawback and the separation of the radial basis from the circular harmonic basis for a free selection of the orthogonal radial polynomials, are presented. Performance measures for the orthogonal moments are discussed from the point of view of image analysis. A moment family list is proposed, which includes most of the representative moments in use and the discrete orthogonal moments.

    1.  Introduction

    Image geometrical moments are region-based integral features useful for describing the image. The moment invariants are combinations of the geometrical moments. They are invariant to scaling, rotation, and other distortions of the image. The orthogonal moments use orthogonal polynomials as the kernel functions in the moment integral. The image moments, moment variants, and orthogonal moments are basic image descriptors and extensively employed for image analysis, shape description, image indexing, annotation, retrieval, registration, matching, object detection, biometric recognition, high level vehicle navigation, image authentication, and image watermarking. Compared with other image descriptors, the moment-based descriptors have rigorous mathematical bases and show the best performance among the low dimensional descriptors for two-dimensional (2D) images[1].

    The first moment invariants have been introduced by Hu[2] more than half century ago. After a long period of quiet development in the image moment theories, the research on the moment-based methods and their applications found a fast growing period since the 1990s with thousands of journal papers and book chapters published each year in the last ten years. A number of new invariant moments and orthogonal moments have been introduced, showing better performance for image description. In the same time, an important part of the research effort has been devoted to improving the speed and accuracy of the moment feature calculation.

    In this Review, we give a short historic overview on the development of the image moment theories involving the moment invariants, the orthogonal moments, and the discrete orthogonal moments. It is impossible to cover the whole area of the image moments in a short overview. Moreover, several excellent textbooks on the image moments theory and applications have been recently published[36]. We describe several selected representative moments, their performance, and the ideas behind their development. We analyze the information suppression drawback in the Zernike moments, which was not addressed in a recent and more complete overview[7]. We present a moment family table. We also propose new criteria for assessing performance among the numerous orthogonal polynomials. This Review, however, does not cover issues of the computation speed and numerical stability of the image moments nor their applications.

    2.  Moment invariants

    The geometrical moments of a 2D image f(x,y) in the rectangular coordinate system are defined as mp,q=xpyqf(x,y)dxdy,where the moment orders p and q are positive integers. Thus, the zero order moment represents the image’s total energy. The first order moments specify the location of the image centroid. The moments computed in the coordinate system with the origin placed at the image centroid are the central moments, which are then translation invariant. The second order moments describe the rectangularity and ellipticity of the image and are used to determine the principle axes of the image[2,8]. The low-order moment features are directly useful for the shape description in the object detection tasks[9].

    A.  Hu’s moment invariants

    The moment invariants first introduced by Hu are mostly known as a set of seven parameters, which are invariant to rotation, scaling, and translation of the image[2]. Hu’s moment invariants are algebraic combinations of the geometrical moments of up to second and third orders obtained by the algebraic invariants theory as h1=μ20+μ02,h2=(μ20μ02)2+4μ112,h3=(μ303μ12)2+(3μ21μ03)2,h4=(μ30+μ12)2+(μ21+μ03)2,h5=(μ303μ12)(μ30+μ12)[(μ30+μ12)23(μ21+μ03)2]+(3μ21μ03)(μ21+μ03)[3(μ30+μ12)2(μ21+μ03)2],h6=(μ20μ02)[(μ30+μ12)2(μ21+μ03)2]+4μ11(μ30+μ12)(μ21+μ03),h7=(3μ21μ03)(μ30+μ12)[(μ30+μ12)23(μ21+μ03)2](μ303μ12)(μ21+μ03)[3(μ30+μ12)2(μ21+μ03)2],where μp,q is the central moments. The affine transform invariant moments were also later derived from the algebraic invariant theory[10]. Hu’s moment invariants are highly concentrated image features and are extensively employed for a long period of time. However, Hu’s moment invariants composed from the low-order geometric moments do not describe the details of the image and suffer from information redundancy and information suppression drawbacks, as first pointed out in Ref. [11]. Hu’s moment invariants are information redundant because the kernels xpyq in their component moments are not orthogonal. As a consequence, an assessment of the quality of the image description by Hu’s moment invariants is not straightforward. One does not know how many Hu’s moment invariants should be used and how well they describe the image for a given task. The information redundancy can be removed by using the orthogonal moments, as will be discussed in Section 3.

    B.  Fourier-Mellin moments

    The information suppression and loss in Hu’s moment invariants may be shown in evidence by the mathematical equivalence between Hu’s moment invariants and the Fourier-Mellin moments. In the polar coordinate system, the Fourier-Mellin moments of an image f(r,θ) are defined as Mn,m=02π01f(r,θ)rn1ejmθrdrdθ,where the radial coordinate r is normalized for the scale invariance, the positive integer n1 is the radial moment order, and the integer m is the circular harmonic order. A rotation of the image corresponds to a translation of the image along the angular coordinate and a phase shift of the circular Fourier transform, whose intensity is rotation invariant.

    In fact, Hu’s moment invariants can be derived more easily in the polar coordinate system as the angular moments[12] or the Fourier-Mellin moments[13,14]. It can be shown[13] that Hu’s moment invariants are equivalent to a special set of the Fourier-Mellin moments with the circular harmonic orders m=03 and the radial moment order n=2, 3. It is well known that the low circular harmonic orders m=03 do not describe angular variations in detail. In general, the circular harmonic orders m=0,1,,M, up to M>10 are required to reconstruct an image from its circular harmonic functions with acceptable detail[15]. On the other hand, the high-order monomials rn suppresses contribution to the moments from the central part of the image and heavily weights the contribution of the background outside the object in the image. The information suppression drawback of the moment invariants can be overcame by using the Fourier-Mellin moments of low radial moment orders.

    C.  Complex moments

    The complex moments are defined in the Cartesian coordinate system but with the complex coordinate kernels (x+jy) and (xjy) instead of x and y, as in Eq. (1)[11,16]. The Fourier-Mellin moments in the polar coordinate system are equivalent to the complex moments in the rectangular coordinate system but with some fractional half-integer orders as Mn,m=02π0f(r,θ)rn1ejmθrdrdθ=f(x,y)(x+jy)(nm)/2(xjy)(n+m)/2dxdy,so that the Fouirer-Mellin moments can be computed in the rectangular coordinate system via the complex moments without transforming the image from the Cartesian coordinate system to the polar coordinate system. The complex moments were also used to derive the affine invariant moments[4].

    D.  Wavelet moments

    The wavelet moments are defined in the polar coordinate system as[17,18]Fa,b,m=02π01ψa,b(r)ejmθf(r,θ)rdrdθ,where the circular Fourier transform gives the rotation invariance, and the wavelet transform is applied in the radial coordinate with the mother wavelets such as the Gabor wavelet, Mexican-hat wavelet, or the cubic B-spline function, the latter is shown as ψ(r)=4αn+12π(n+1)σcos(2πf0(2r1))exp((2r1)22σ2(n+1)),where n=3, α=0.697, f0=0.409, and σ2=0.561. The wavelet transform is a convolution of the image’s circular harmonic function fm(r)=02πejmθf(r,θ)dθ,with the wavelets, which are scaled, and translated mother wavelets that are shown as ψa,b(r)=1aψ(rba),where the discrete scaling factor is a=(1/2)p, the discrete translation factor is b=(q/2)(1/2)p, and p and q are positive integers. The continuous wavelets such as the cubic B-spline functions are bandpass filters[19]. By adaptively choosing the appropriate scale a, the wavelet transform can highlight the local features such as edges in the image. However, the wavelet moments as defined in Eq. (5) are not orthogonal and, therefore, can be used as image features for the image description but not for the image reconstruction. Moreover, the wavelet moments are not scale invariant. The image must be resized to a fixed size before computing the wavelet moments[17,18].

    3.  Orthogonal moments

    The orthogonal moments are an expansion of the image into the orthogonal polynomial bases. The orthogonal moments are information independent from each other, allowing the image reconstruction for assessing the quality of the image description.

    A.  Rotation-invariant orthogonal moments

    The orthogonal polynomials result from the power series solutions of the ordinary differential equations with the orthogonality of the polynomials related to the Sturm-Liouville forms of the differential equations plus the boundary conditions. They are referred to as the special functions and are given special names because of their frequent uses. On the other hand, the orthogonal polynomials can be easily generated using the Gram-Schmidt orthonormalization, so that there can be an infinite number of orthogonal polynomials. The well-known orthogonal polynomials include the Jacobi, Hermite, Laguerre, and Bessel polynomials. The interrelations between the polynomials exist. For instance, the Gegenbauer, Legendre, Chebyshev, and Zernike polynomials are special cases of the Jacobi polynomials, which are the solutions of the hypergeometric differential equations.

    Many orthogonal polynomials are proposed as the bases for the image analysis. For the 2D expansion of an image in the polar coordinate system, the radial orthogonal polynomials are the radial basis, and the Fourier kernels exp(jmθ) are used for the circular harmonic analysis. As pointed out by Bhatia and Wolf[20,21], for the rotation invariance in the form, the circular Fourier kernel is a unique solution to be included as the basis function. The scale invariance is achieved by using the normalized radial coordinate.

    1.  Zernike and Pseudo-Zernike moments

    The Zernike moments are orthogonal on the unit circle. The Zernike radial polynomials play a dominant role for the aberration characterization in the optical design. The Zernike moments were proposed in 1980 for image analysis[22] as Anm=n+1π02π01Rnm(r)exp(jmθ)f(r,θ)rdrdθ,where the Zernike radial polynomials are Rnm(r)=s=0(n|m|)/2(1)s(ns)!s!((n+|m|)/2s)!((n|m|)/2s)!rn2s,where m is the circular harmonic order, and n is the degree of the Zernike polynomial. The Zernike moments are designed to be presented in the Cartesian coordinate system, so that the Zernike radial polynomials must not contain powers of r smaller than the circular harmonic order, n|m| and n|m| are even[20]. In the Zernike moments, each term of the radial power in Rnm(r) corresponds to a radial moment of a specific order, so that the Zernike moments can be computed as the algebraic combinations of the Fourier-Mellin moments.

    Bhatia and Wolf derived[20] another orthogonal set of polynomials in x and y, which contains only the powers of r higher than circular harmonic order n|m|, but with the condition for even n|m| removed[20]. This moment set is now referred to as the pseudo-Zernike moments[23]. When high circular harmonics of the order m are used to represent the image of the angular variation in the range [0,2π][15], the Zernike and pseudo-Zernike moments with the radial polynomial degrees n|m| must suffer from severe information suppression drawback by the high radial moment orders.

    2.  Orthogonal Fourier-Mellin moments

    The orthogonal Fourier-Mellin moments[24] are designed to avoid the information suppression issue in the Zernike moments. The orthogonal Fourier-Mellin moments are defined as Φnm=12πan02π01Qn(r)exp(jmθ)f(r,θ)rdrdθ,where an=½(n+1), and the radial polynomials Qn(r)=s=0nαnsrs,withαns=(1)n+s(n+s+1)!(ns)!s!(s+1)!.are obtained by the Gram-Schmidt orthogonalization of the power series rn with n=0,1,2,. The orthogonal Fourier-Mellin moments do not need to be expressed in the Cartesian coordinate system. Hence, the restriction of n|m| in the Zernike and pseudo-Zernike moments is removed. This removal of the restriction improves the performance in the image analysis.

    The Fourier-Mellin polynomials Qn(r) have n zeros in 0r1, while the Zernike polynomials have (nm)/2 duplicated roots for Rnm(r)=0. To have the same number n of zeros, the degree of the Zernike polynomial must be as high as 2n+|m|. Figure 1 shows the Zernike polynomials Rnm(r) for the circular harmonic order m=10 and the degree n=1028 with the zeros distributed in the range of radial distance r0.2r1. The closest endpoint to the origin of R28,10(r) is at r=0.2. The image information in the region of the normalized radial distance r0.2 is then completely suppressed in this set of Zernike moments. The set of orthogonal Fourier-Mellin radial polynomials Qn(r) with the degree n=09, independent of m, contain oscillations with the zeros distributed over the radial coordinate range of 0.04r1, as shown in Fig. 1(b). Thus, the set of orthogonal Fourier-Mellin moments favors the analysis of the image, especially for the central part of the image or the small scale object in the unit circle, which occurs in the context of the scale invariant pattern recognition. Figure 2 shows a capital letter E reconstructed from a maximum of 64 orthogonal Fourier-Mellin moments and those from the Zernike moments. The former in Fig. 2(c) shows much better detail than the latter in Fig. 2(b).

    Orthogonal radial polynomials: (a) Zernike polynomials with the degrees n=10–28 for circular harmonic order m=10, (b) orthogonal Fourier-Mellin polynomials with the degrees n=0–9.

    Figure 1.Orthogonal radial polynomials: (a) Zernike polynomials with the degrees n=1028 for circular harmonic order m=10, (b) orthogonal Fourier-Mellin polynomials with the degrees n=09.

    (a) Original image of a letter E in the unit disk; (b) reconstructed from first 64 orthogonal Fourier-Mellin moments Φn,m with n, m=0–7; (c) reconstructed from Zernike moments Rn,m with circular harmonic orders m=0–7 and for each m using the eight lowest degrees, satisfying n≥|m|+2.

    Figure 2.(a) Original image of a letter E in the unit disk; (b) reconstructed from first 64 orthogonal Fourier-Mellin moments Φn,m with n, m=07; (c) reconstructed from Zernike moments Rn,m with circular harmonic orders m=07 and for each m using the eight lowest degrees, satisfying n|m|+2.

    The key idea behind the design of the orthogonal Fourier-Mellin moments is to completely separate the radial polynomials from the circular Fourier kernel, and to avoid the constraint that the radial moment orders should be higher than the circular harmonic orders. Based on this idea, many new orthogonal radial polynomials can be proposed, such as Chebyshev-Fourier, Jacobi-Fourier, and Bessel-Fourier polynomials. All the moments of this type can be computed as algebraic combinations of the Fourier-Mellin moments. The latter can be computed from the equivalent complex moments as shown in Eq. (4) without the transformation of the image from the rectangular to the polar coordinate system.

    3.  Chebyshev-Fourier moments

    The shifted Chebyshev polynomial of the second kind, which is orthogonal in the interval [0,1], is combined with the circular Fourier kernel to form the kernel of the Chebyshev-Fourier moments as[25]ϕnm=12π02π01Rn(r)exp(jmθ)f(r,θ)rdrdθ,with n=0,1,2,m=0,±1,±2, and Rn(r)=8π[w(r)]1/2r1/2Un*(r),with shifted Chebyshev polynomials of the second kind Un*(r)=k=0[n/2](1)k(nk)!k!(n2k)![2(2r1)]n2k,which are orthogonal over the range 0r1 as 01Un*(r)Uk*(r)w(r)dr=π8δnk,where δnk is the Kronecker symbol, and w(r)=(rr2)1/2 is the weight function. As shown in Fig. 3, Rn(r) possesses n zero points evenly distributed over the interval [0,1]. The oscillation amplitude of the Rn(r) of various degrees n are close to constant in a large portion of the radial interval [0,1], except that when r tends to be zero, Rn(r) tends to be high absolute values, as shown in Fig. 3. This can be a potential drawback. Experiments showed[25] that the performance for describing an image of the Chebyshev-Fourier moments is similar as that of the orthogonal Fourier-Mellin moments[25].

    Shifted Chebyshev polynomial Rn(r) with n=1, 2, 9, 10, [25]

    Figure 3.Shifted Chebyshev polynomial Rn(r) with n=1, 2, 9, 10, [25]

    4.  Generic Jacobi-Fourier moments

    The Jacobi-Fourier moments are referred to as the generic orthogonal moments[26] with the kernel composed of the radial Jacobi polynomial and the angular Fourier complex exponential kernel, and are defined as Φnm=12π02π01Jn(p,q,r)exp(jmθ)f(r,θ)rdrdθ,where the radial Jacobi polynomials are generated by the orthogonalization of the natural power sequence of the radial coordinate with the weighting function[20,27] as w(p,q,r)=(1r)pqrq1,where pq>1,q>0, such as 01Gn(p,q,r)Gk(p,q,r)w(p,q,r)dr=bn(p,q)δnk,and the radial Jacobi polynomials are Gn(p,q,r)=n!Γ(q)Γ(n+p)s=0n(1)nsΓ(n+p+s)(ns)!s!Γ(q+s)rs,which become, after the normalization, Jn(p,q,r)=Gn(p,q,r)w(p,q,r)/bn(p,q,r).where the normalization constant is bn(p,q)=n!Γ(n+pq+1)[Γ(q)]2(2n+p)Γ(n+p)Γ(n+q).It can be shown that the Jacobi-Fourier moment corresponds to the Legendre-Fourier moments with p=q=1 and the unit weighting function, the Chebyshev-Fourier moments with p=2 and q=3/2, and the orthogonal Fourier-Mellin moments with p=q=2 and the weighting function equal to r[26]. The Zernike radial polynomial corresponds to the Jacobi radial polynomial as rmJk(p,q,r2) with p=q=m+1, where m is the circular harmonic order in the Zernike moments, and k=(nm)/2. The pseudo-Zernike radial polynomial corresponds to the Jacobi radial polynomial Jk(p,q,r) with p=2m+2, q=m+2 and k=nm[3]. Therefore, the Jacobi-Fourier moments provide a generic framework to express the rotation-invariant orthogonal moments. The common formulation is a benefit for the performance comparison and analysis of the orthogonal moments and for searching prime orthogonal moments.

    5.  Bessel-Fourier moments

    The Bessel-Fourier moments were introduced recently, which are defined as[28]Bnm=12παn02π01Jv(λnr)exp(jmθ)f(r,θ)rdrdθ,where Jv(λnr) is the Bessel radial polynomials of order v of the first kind, and λn is the nth zero of Jv(λn)=0. The Bessel polynomials of the same order v are orthogonal over the radial interval [0,1], such as 01Jv(λnr)Jv(λkr)dr=αnδnk,where αn=[Jv+1(λn)]2/2 is the normalization constant. Figure 4 is a plot of the Bessel function v=1. The Bessel polynomials of the first kind J1(λnr) tend to be zero at both ends of the radial interval [0,1] and show a slow variation of the oscillation amplitude over the radial range [0,1], as shown in Fig. 4. Only in the range of r close to r=0, the oscillation amplitudes are doubled.

    Bessel Radial polynomial J1(λnr) with n=0,1,…,9, [28].

    Figure 4.Bessel Radial polynomial J1(λnr) with n=0,1,,9, [28].

    Compared to the orthogonal Fourier-Mellin polynomials and the shifted Chebyshev polynomials, both of which tend to have high values when r tends to be zero, as shown in Figs. 1 and 3, the Bessel-Fourier moments show a better performance for image analysis. They also are better than the Zernike moments, as the Zernike moments show a sharp increase to the unit when r tends to be r=1, as shown in Fig. 1(a).

    6.  Radial-Harmonic-Fourier transform

    The radial-harmonic-Fourier moments[29], or polar harmonic transform[30,31], includes the Fourier harmonic analysis in both the radial and angular coordinates and are defined as ϕnm=12π02π01Tn(r)exp(jmθ)f(r,θ)rdrdθ.where the radial kernels Tn(r) are the orthogonal triangular functions, such as Tn(r)=sin[(n+1)πr] for the odd n, and Tn(r)=cos(nπr) for the even n, or the complex exponential functions such as Tn(r)=exp(j2πnr2) or Tn(r)=exp(jnr). In all cases, the Tn(r) should be normalized, respectively. The corresponding transforms are referred to as the radial-harmonic-Fourier moments[29], the polar harmonic transform[30], and the exponent-Fourier moments[31]. The angular radial transform, the polar complex exponential transform, the polar cosine transform, and the polar sine transform also belong to this group[3133]. As the Fourier kernels have uniform distributions of the zeros and a constant oscillation amplitude over the interval 0r1 and 0θ2π, this type of transform shows a superior performance over the other rotation-invariant orthogonal moments with radial polynomial kernels[34].

    The radial-harmonic-Fourier moments are not scale invariant. An image must be rescaled to a given size before computing the moments. Moreover, the radial-harmonic-Fourier moments should preferably be referred to as the radial-harmonic-Fourier transforms, as the moments are essentially formulated with the monomial and polynomial kernels. Moreover, the moments with the radial polynomials can achieve the scale invariance by using the normalized radial coordinate.

    B.  Orthogonal moments in the Cartesian coordinate system

    There are an infinite number of complete sets of orthogonal polynomials. Many of them were proposed as the kernels of the orthogonal moments for 2D images in the Cartesian coordinate system. The orthogonal moments in this category include the Legendre moments[22,35], the Gegenbauer moments[36,37], the Gaussian-Hermite moments[3840], etc. Defined in the Cartesian coordinate system, these moments enjoy the orthogonality of the orthogonal polynomials in x and y in the interval [1,1] or [0,1] for image analysis and reconstruction. They can also be scale invariant when using the scaling factor to normalize the coordinates.

    A new type of the orthogonal moments uses two different polynomials as the basis for each dimension in the Cartesian coordinate system separately, such as the Chebyshev-Gegenbauer and the Gegenbauer-Legendre moments. These moments are called separable moments[41,42].

    The orthogonal moments in the Cartesian coordinate system are not rotational invariant by definition. However, the rotation invariance can still be introduced by the transformation between the Cartesian and polar coordinate systems.

    C.  Performance measures of orthogonal moments

    Performances of the orthogonal moment image descriptors have been assessed in many Letters via the image reconstruction errors and the pattern recognition scores. However, these merits are dependent on the selected moment orders and the testing image, and therefore are the indirect measures. In fact, as the orthogonal moments are the expansions of the image into the orthogonal polynomial bases, the performance of the image description by the orthogonal moments is determined directly by the bases’ functions. In the case of rotation-invariant orthogonal moments, the Fourier harmonic analysis is applied along the angular coordinate. Hence, the performance of the image analysis along the radial coordinate depends uniquely on the set of the radial orthogonal polynomials. For the image analysis, the radial polynomial set should have a high number of real-valued zeros, a wide range of the distribution of zeros in the whole range 0r1, and a constant oscillation amplitude. From instance, the Zernike radial polynomials are restricted to having high degrees, and one set of Zernike moments, shown in Fig. 1(a), have the zeros distributed in the range of 0.2r1. The orthogonal Fourier-Mellin polynomials and shifted Chebyshev polynomials both tend to have high values when r tends to be zero, as shown in Figs. 1(b) and 3. Those facts represent the weakness of these moments. The Bessel polynomials have n+2 zeros distributed in the range of 0r1, and a slow variation of the oscillation amplitudes, as shown in Fig. 4, so that they can have a better performance for the image analysis. In this sense, the radial-harmonic-Fourier transforms have the best performance, as this is simply the Fourier harmonic analysis in both the angular and radial dimensions. However, they are not image moments in the mathematical sense and are not scale invariant.

    Furthermore, in terms of the space-frequency space analysis, the Fourier kernels, which are perfectly local in frequency but are global in space, are not efficient for describing local details in the image. The Fourier moments miss the location of an image detail, and very high frequency orders must be involved to reconstruct a sharp detail, such as an edge in the image. In this sense, the wavelet moment using the Gabor transform and the wavelet transform can be superior over the Fourier transform[43]. However, the wavelet moments using the continuous mother wavelets are not orthogonal.

    Table 1 shows a family of the moments with the classification. However, it is regrettable to not be able to list all image moments in Table 1 in the period of time when many novel moments are being proposed. For a given image description task, the choice among the available moments should depend on the specific tasks, the requested invariance, the computation performance, and many other considerations.

    D.  Discrete orthogonal moments

    Most orthogonal moments discussed previously belong to continuous moments, as their bases are continuous functions with their orthogonality defined by the continuous integrals. Numerical computation of the continuous moments requires discretization of the continuous polynomials and discrete approximation of the continuous moment integrals, which causes the errors, affecting the accuracy of the calculation and the quality of image reconstruction with increased computational complexity.

    Non-orthogonal momentsMoment invariants (Hu’s) Fourier-Mellin moments complex moments wavelet moments
    Orthogonal momentsContinuous orthogonal momentsCartesian momentsLegendre moments Gaussian-Hermite moments Gegenbauer moments
    Circular momentsZernike moments pseudo-Zernike moments orthogonal Fourier-Mellin moments Chebyshev-Fourier moments Jacobi-Fourier moments Bessel-Fourier moments radial-harmonic-Fourier transform
    Discrete orthogonal momentsCartesian momentsTchebichef moments Krawtchouk moments Hahn moments dual Hahn moments Racah moments
    Circular momentsRadial Tchebichef moments radial Krawtchouk moments

    Table 1. Family of Image Moments

    The discrete orthogonal moments[4455] use discrete orthogonal polynomials as the basis set, which satisfies the orthogonality exactly without numerical approximation. For a smooth image intensity distribution, whose value is known only at a set of discrete coordinate points, the discrete polynomials pn(x) are also defined on these distinct real nodes, and satisfy the discrete orthogonality relation as x=0N1pm(x)pn(x)w(x)=ρ(n)δmn,where 0m,nN1, and ρ(n) is the square norm of pn(x). The weight w(x) in the orthogonality expression (26) of the discrete polynomials can be the Meixner, Charlier, Krawtchouk, and Hahn weight. Thus, the discrete orthogonal polynomials are uniquely determined by the given distinct nodes support, the weights, and the orthogonality condition, and can be built from monomials by the Gram-Schmidt process.

    The discrete orthogonal moments of a 2D image f(x,y) in the Cartesian coordinate system on a closed interval are defined as Pmn=x=0M1y=0N1f(x,y)p˜m(x)p˜n(y),where the normalized discrete polynomial bases p˜n(x) are shown as p˜n(x)=pn(x)w(x)/ρ(n).As the discrete polynomials satisfy the orthogonal property precisely, no approximation errors are involved in the computation of moments. The image can be accurately reconstructed from the complete set of discrete moments. When two different discrete orthogonal polynomials serve as the bases in x and y, separately, the moments are separable moments such as that using the Tchebichef-Krawtchouk, Hahn-Krawtchouk, and Charlier-Hahn polynomials combined with Tchebichef, Krawtchouk, Hahn, or Charlier polynomials[41,42].

    1.  Discrete orthogonal Tchebichef moments

    The first discrete orthogonal Tchebichef (Chebyshev) moments[44] were proposed based on the discrete orthogonal Tchebichef polynomials. The discrete Tchebichef polynomials are shown as tn(x)=n!k=0n(1)nk(N1knk)(n+kn)(xk),satisfying the orthogonality condition with the unit weight and the normalization constants ρ(n,N)=N(N21)(N222)(N2n2)2n+1=(2n)!(N+n2n+1),n=0,1,N1.The normalized Tchebichef polynomials are t˜n(x)=tn(x)/β(n,N),with the normalized constant ρ˜(n,N)=ρ(n,N)/β(n,N)2andβ(n,N)=Nn.The discrete Tchebichef moments exhibit a superior performance for image reconstruction and feature representation compared to the continuous Legendre and Zernike moments[44].

    2.  Discrete orthogonal Krawtchouk moments

    One of representative discrete orthogonal moments defined in the Cartesian coordinate system is the Krawtchouk[46] moment, which are based on the Krawtchouk polynomials. The kernel functions are the separable discrete orthogonal polynomials in x and y. The discrete Krawtchouk polynomials satisfy the orthogonality conditions with the Krawtchouk weight as w(x;p,N)=(Nx)px(1p)Nx,and the normalization constant ρ(n;p,N)=(1)n(1pp)nn!(N)n.The Krawtchouk polynomials are obtained as Kn(x;p,N)=k=0Nak,n,pxk=F12(n,x;N;1p),where x,n=0,1,2,N, N>0, and p(0,1), the Frs is the hypergeometric series, defined as Frs(a1,,ar;b1,,bs;z)=k=0(a1)k(a2)k(ar)k(b1)k(b1)k(bs)kzkk!,and (a)k is the Pochhammer symbol given by (a)k=a(a+1)(a+k1)=Γ(a+k)Γ(a).The Krawtchouk moments perform better than the Tchebichef moments in terms of the reconstruction error. They were employed to extract local features of an image by varying the parameter of the binomial distribution associated with the Krawtchouk polynomials[46].

    3.  Discrete orthogonal Hahn moments

    Two types of Hahn polynomials with different standardizations were used to define two types of Hahn moments[47,48], respectively. The Hahn polynomials are expressed as hn(μ,ν)(x,N)=(N+ν1)n(N1)n×k=0n(1)k(n)k(x)k(2N+μ+νn1)k(N+ν1)k(N1)k1k!,where μ,ν(μ>1,ν>1) are the adjustable parameters controlling the shape of the polynomials. The Hahn polynomials satisfy the discrete orthogonal condition as x=0N1w(x)hm(μ,ν)(x,N)hn(μ,ν)(x,N)=dn2δmn,with the weighting function as w(x)=1Γ(x+1)Γ(x+μ+1)Γ(N+νx)Γ(Nnx),and the square norm dn2 has the following expression dn2=Γ(2N+μ+νn)(2N+μ+ν2n1)Γ(N+μ+νn)×1Γ(N+μn)Γ(N+νn)Γ(n+1)Γ(Nn).Compared with the Tchebichef moments and the Krawtchouk moments, the first type of Hahn moments has better image reconstruction results[47]. Another type of Hahn moments[48] is based on another form of Hahn polynomials, shown as hn(x;α,β,N)=F23(n,n+α+β+1,x;α+1,N;1),where x,n=0,1,2,,N1, α>1,β>1, or α<N, β<N, with the weighting function w(x;α,β,N)=(α+xx)(β+NxNx),and the square norm d2(n;α,β,N)=(1)n(n+α+β+1)N+1(β+1)nn!(2n+α+β+1)(α+1)n(N)nN!.This type of Hahn moments provides a unified understanding of the Tchebichef and Krawtchouk moments. The two latter moments can be obtained as particular cases of the Hahn moments with the appropriate parameter settings. This fact implies that the Hahn moments encompass all their properties and exhibit intermediate properties between the extremes set by the Tchebichef and Krawtchouk moments. Besides, the Hahn moments, as a generalization of the Tchebichef and Krawtchouk moments, can be used for global and local feature extraction[48].

    4.  Discrete orthogonal dual Hahn moments

    The discrete Tchebichef, Krawtchouk, and Hahn polynomials are orthogonal on a uniform lattice {x=0,1,2,}. There is another category of discrete variable polynomials, which are orthogonal on a non-uniform lattice {x=x(s),s=0,1,2,}. For different non-uniform lattice functions, there are different polynomials. The dual Hahn polynomials and the Racah polynomials are two types of discrete polynomials orthogonal on a non-uniform lattice x(s)=s(s+1). Due to the finite domain of the definition, the dual Hahn polynomials and Racah polynomials were explored to construct the corresponding dual Hahn moments[50] and Racah moments[52]. They are defined in the Cartesian coordinate system, and their kernel functions are the product of two corresponding two corresponding 1D separable discrete orthogonal polynomials.

    Given a uniform pixel lattice image f(s,t) with a size N×N, the dual Hahn moments[50] are defined as Wnm=s=ab1t=ab1w^n(c)(s,a,b)w^m(c)(t,a,b)f(s,t),with n,m=0,1,2,,N1, and the dual Hahn polynomials wn(c)(s,a,b) defined as wn(c)(s,a,b)=(ab+1)n(a+c+1)nn!×F23(n,as,a+s+1;ab+1,a+c+1;1),where parameters a, b, and c are restricted to (1/2)<a<b, |c|<1+a,b=a+N. The dual Hahn polynomials satisfy the following orthogonality property s=ab1ρ(s)wn(c)(s,a,b)wm(c)(s,a,b)[Δx(s12)]=dn2δnm.where ρ(s) is the weighting function ρ(s)=Γ(a+s+1)Γ(c+s+1)Γ(sa+1)Γ(bs)Γ(b+s+1)Γ(sc+1),and dn2 is the square norm dn2=Γ(a+c+n+1)n!(ban1)!Γ(bcn).The normalized dual Hahn polynomials are w^n(c)(s,a,b)=wn(c)(s,a,b)ρ(s)dn2[Δx(s12)].The Tchebichef polynomials, Krawtchouk polynomials, and Hahn polynomials are special cases of the dual Hahn polynomials. Thus, the dual Hahn moments which contain more parameters have a generality and give more flexibility to promote the image describing ability. The dual Hahn moments perform better than the Legendre moments, Tchebichef moments, and Krawtchouk moments in terms of image reconstruction capability[50].

    5.  Discrete orthogonal Racah moments

    Similar to the dual Hahn moments, for the image f(s,t) with a size N×N, the Racah moments[52] are defined as Unm=s=ab1t=ab1u^n(α,β)(s,a,b)u^m(α,β)(t,a,b)f(s,t),with n,m=0,1,2,,N1. Racah polynomials un(α,β)(s,a,b) are defined as un(α,β)(s,a,b)=1n!(ab+1)n(β+1)n(a+b+α+1)n×F34(n,α+β+n+1,as,a+s+1;β+1,ab+1,a+b+α+1;1),where the parameters a,b,α and β are restricted by (1/2)<a<b,α>1,1<β<2a+1,b=a+N.The Racah polynomials satisfy the following orthogonality property s=ab1ρ(s)un(α,β)(s,a,b)um(α,β)(s,a,b)[Δx(s12)]=dn2δnm,with the weight ρ(s)=Γ(a+s+1)Γ(sa+β+1)Γ(b+αs)Γ(b+α+s+1)Γ(aβ+s+1)Γ(sa+1)Γ(bs)Γ(b+s+1),and the square norm dn2=Γ(α+n+1)Γ(β+n+1)Γ(ba+α+β+n+1)(α+β+2n+1)n!(ban1)!Γ(α+β+n+1)×Γ(a+b+α+n+1)Γ(a+bβn).The normalized Racah polynomials are u^n(α,β)(s,a,b)=un(α,β)(s,a,b)ρ(s)dn2[Δx(s12)].The Racah moments perform better than the Legendre moments, Tchebichef moments, and Krawtchouk moments for noise-free image reconstruction[52].

    6.  Rotation-invariant discrete orthogonal moments

    Rotation-invariant discrete radial Tchebichef moments defined in polar coordinate systems were proposed based on the 1D radial discrete Tchebichef polynomials and the circular Fourier kernel[53]. The primary advantage of the discrete radial moments is the rotational invariance. For an image f(r,θk), the explicit expressions of the radial Tchebichef moments[53] are as follows: Spq=1nρ(p,m)r=0m1k=0n1tp,m(r)ej2πqknf(r,θk),where r=0,1,,m1, m=N/2,θk=2πk/n, and k=0,1,,n1, with n denoting the maximum number of pixels along the circumference of the circle of radius m, tp,m(r)=p!mpk=0p(1)pk(m1kpk)(p+kp)(rk),ρ(p,m)=m(11m2)(122m2)(1p2m2)2p+1,with p=0,1,m1. The rotation-invariant discrete radial Krawtchouk moments[54] were introduced in a similar way to the radial Tchebichef moments. The kernel functions of the radial Krawtchouk moments are products of radial 1D Krawtchouk polynomials and angular circular Fourier functions.

    4.  Conclusion

    We have reviewed some major steps in the development of the image moment theory, involving the geometrical moments, the moment invariants, and the orthogonal moments. It is impossible for this short Review to cover all the important moments in detail. Moreover, we have not included in this Review the works on the speed and accuracy of computation of the image moments and on their applications. A large body of excellent Letters and text books exist in the literature now. A more rigorous, accurate, and complete mathematical analysis on the image moments theory has been developed. However, some fundamental intuitive ideas, such as the use of low-order radial moments for avoiding information suppression, the definition of the moment invariants in the polar coordinate system and relation of them to the moments in the rectangular coordinate system via the complex moments, and the separation of the radial moment orders from the circular harmonic orders, have been widely accepted and continuously used in the development of the image moment-based methods. We proposed a moment family table and the performance measures for the orthogonal moments based on the number of zeros, the uniform distribution of zeros, and the oscillation amplitude variation of the orthogonal polynomials. The image reconstruction can depend on the choice of moment orders and on the testing image set so that they can be indirect measures for the performance.

    References

    [1] K. Mikolajczyk, C. Schmid. IEEE Trans. Pattern Anal. Mach. Intell., 27, 1615(2005).

    [2] M. K. Hu. IRE Trans. Inform. Theor., IT-8, 179(1962).

    [3] G. A. Papakostas. Moments and Moment Invariants—Theory and Applications(2014).

    [4] J. Flusser, T. Suk, B. Zitová. Moments and Moment Invariants in Pattern Recognition(2009).

    [5] R. Mukundan, K. Ramakrishnan. Moment Functions in Image Analysis—Theory and Applications(1998).

    [6] M. Pawlak. Image Analysis by Moments: Reconstruction and Computational Aspects(2006).

    [7] G. A. Papakostas. Over 50 Years of Image Moments and Moment Invariants(2014).

    [8] Y. Sheng, H. Arsenault. J. Opt. Soc. Am. A, 4, 1176(1987).

    [9] S. A. Dudani, K. J. Breeding. IEEE Trans. Comput., c-26, 39(1977).

    [10] J. Flusser, T. Suk. Pattern. Recognit., 26, 167(1993).

    [11] Y. S. Abu-Mostafa, D. Psaltis. IEEE Trans. Pattern Anal. Mach. Intell., PAMI-6, 698(1984).

    [12] J. F. Boyce, W. J. Hossack. Pattern. Recognit. Lett., 1, 451(1983).

    [13] Y. Sheng, J. Duvernoy. J. Opt. Soc. Am. A, 3, 885(1986).

    [14] Y. Sheng, H. H. Arsenault. J. Opt. Soc. Am. A, 3, 771(1986).

    [15] H. H. Arsenault, Y. Sheng. Appl. Opt., 25, 3225(1986).

    [16] Y. S. Abu-Mostafa, D. Psaltis. IEEE Trans. Pattern Anal. Mach. Intell., PAMI-7, 46(1985).

    [17] D. Shen, H. H. Ip. Pattern Recognit., 32, 151(1999).

    [18] X. Hu, B. Kong, F. Zheng, S. Wang. International Conference on Information Acquisition(2007).

    [19] H. Szu, Y. Sheng, J. Chen. Appl. Opt., 31, 3267(1992).

    [20] A. B. Bhatia, E. Wolf. Mathematical Proceedings of the Cambridge Philosophical Society, 50, 40(1954).

    [21] M. Born, E. Wolf. Principles of Optics(1975).

    [22] M. R. Teague. J. Opt. Soc. Am., 70, 920(1980).

    [23] C. H. Teh, R. T. Chin. IEEE Trans. Pattern Anal. Mach. Intell., 10, 496(1988).

    [24] Y. Sheng, L. Shen. J. Opt. Soc. Am. A, 11, 1748(1994).

    [25] Z. Ping, R. Wu, Y. Sheng. J. Opt. Soc. Am. A, 19, 1748(2002).

    [26] Z. L. Ping, H. P. Ren, J. Zou, Y. L. Sheng, W. Bo. Pattern. Recognit., 40, 1245(2007).

    [27] M. Abramowitz, I. A. Stegun. Handbook of Mathematical Functions: With Formulas, Graphs, and Mathematical Tables(1964).

    [28] B. Xiao, J.-F. Ma, X. Wang. Pattern. Recognit., 43, 2620(2010).

    [29] H. Ren, Z. Ping, W. Bo, W. Wu, Y. Sheng. J. Opt. Soc. Am. A, 20, 631(2003).

    [30] P. Yap, X. Jiang, A. C. Kot. IEEE Trans. Pattern Anal. Mach. Intell., 32, 1259(2010).

    [31] T. V. Hoang, S. Tabbone. Proceedings of 18th IEEE International Conference on Image Processing (ICIP), 829(2011).

    [32] H. T. Hu, Y. D. Zhang, C. Shao, Q. Ju. Pattern Recognit., 47, 2596(2014).

    [33] Q. Miao, J. Liu, W. Li, J. Shi, Y. Wang. Opt. Commun., 285, 1044(2012).

    [34] M. Bober. IEEE Trans. Circ. Syst. Video Tech., 11, 716(2001).

    [35] C.-W. Chong, P. Raveendran, R. Mukundan. Pattern. Recognit., 37, 119(2004).

    [36] S. Liao, A. Chiang, L. Qin, M. Pawlak. Proceedings of 16th International Conference on Pattern Recognition, 485(2002).

    [37] K. M. Hosny. Pattern. Recognit. Lett., 32, 795(2011).

    [38] B. Yang, M. Dai. Signal Process., 91, 2290(2011).

    [39] B. Yang, G. Li, H. Zhang, M. Dai. Pattern. Recognit. Lett., 32, 1283(2011).

    [40] L. Geng-Xiang, Y. Bo, D. Mo. Proceedings of International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR), 12(2011).

    [41] A. Hmimid, M. Sayyouri, H. Qjidaa. J. Electron. Imaging, 23, 013026(2014).

    [42] H. Zhu. Pattern Recognit., 45, 1540(2010).

    [43] Y. Sheng. Wavelet transform. Transforms and Applications(2010).

    [44] R. Mukundan, S. H. Ong, P. A. Lee. IEEE Trans. Image Process., 10, 1357(2001).

    [45] H. Zhu, H. Shu, T. Xia, L. Luo, J. Louis Coatrieux. Pattern. Recognit., 40, 2530(2007).

    [46] P. T. Yap, R. Paramesran, O. Seng-Huat. IEEE Trans. Image Process., 12, 1367(2003).

    [47] M. Kamel, J. Zhou, H. Shu, A. Campilho, H. Zhu, C. Toumoulin, L. Luo. Image Analysis and Recognition, 524(2005).

    [48] Y. Pew-Thian, R. Paramesran, O. Seng-Huat. IEEE Trans. Pattern Anal. Mach. Intell., 29, 2057(2007).

    [49] H.-A. Goh, C.-W. Chong, R. Besar, F. S. Abas, K.-S. Sim. Int. J. Image Graph., 9, 271(2009).

    [50] H. Zhu, H. Shu, J. Zhou, L. Luo, J. L. Coatrieux. Pattern. Recognit. Lett., 28, 1688(2007).

    [51] E. G. Karakasis, G. A. Papakostas, D. E. Koulouriotis, V. D. Tourassis. Pattern. Recognit., 46, 1998(2013).

    [52] H. Zhu, H. Shu, J. Liang, L. Luo, J.-L. Coatrieux. Signal Process., 87, 687(2007).

    [53] R. Mukundan. TENCON: 2005 IEEE Region 10, 1(2005).

    [54] P. Ananth raj, A. Venkataramana. Proceedings of 6th International Conference on Information, Communications & Signal, 1(2007).

    [55] H. Zhu, M. Liu, H. Shu, H. Zhang, L. Luo. IET Image Process., 4, 335(2010).

    Kejia Wang, Ziliang Ping, Yunlong Sheng. Development of image invariant moments—a short overview[J]. Chinese Optics Letters, 2016, 14(9): 091001
    Download Citation