An Interactive Multiple Model (IMM) filter algorithm based on Expected-mode Augmentation (EMA) named EMA-IMM algorithm is proposed to overcome the uncertain model and time-varied model parameters of the integrated navigation system for an Autonomous Underwater Vehicle( AUV) in a tough environment. The EMA-IMM algorithm mainly uses the probability of models obtained from the recursive estimate processing for making decision. It filters for the base grids of fixed structure to obtain a fined amendatory model set firstly. Then the amendatory model is filtered to obtain an expected model consisting of a small number of amendatory model grids that are close to the real model. Through a further filtering using the expected model, the suboptimal solution approximate to the real model will be ultimately achieved. Simulation results on the integrated navigation system show that the EMA-IMM algorithm can improve the estimation precisions of longitude and latitude by 97％and 44% respectively as compared with the Kalman filtering algorithm and by 22% and 19% with the IMM algorithm, which proves the superiority of the proposed EMA-IMM algorithm.