For the problem of time-varying multi-target tracking in clutter environment， the Extended Kalman-Gaussian Mixture Probability Hypothesis Density (EK-GMPHD) filter algorithm applicable to non-linear system is presented. In recursion of Gaussian component， the extended Kalman filter is used for local linearization， which solves the non-linear problem of measurement equation and state equation. In reducing the number of Gaussian terms， a new merge criterion is established， which gives consideration to the influence of Gaussian component covariance on the estimation accuracy. The estimated number of targets at the current moment is used to smooth the estimated number of targets at the previous moment， thus can eliminate the influence of outlier. Simulation results show that the proposed algorithm can effectively filter out clutter， and accurately estimate the number and state of multiple targets.