To solve the problem that the traditional vision SLAM is not fully capable of understanding the semantic information, semantic vision SLAM uses semantic landmarks to improve the localization accuracy of the robot.Accurate association of semantic landmarks is the key to deep localization and navigation of the robot, and incorrect association will lead to loss of localization.To address the problem of high association ambiguity caused by dynamic perturbation and observation noise perturbation, a method combining nonparametric clustering with stochastic approximate inference is proposed to improve the accuracy of semantic landmark association, and accurate localization is realized by correct data association.The results of simulations and experiments on KITTI data set show that, the proposed algorithm can improve the accuracy and robustness of data association of semantic landmarks under noise perturbation, fuse the semantic and geometric information to optimize the poses of the robot and semantic landmarks, and improve the localization accuracy of the robot.