Aiming at the problem that the image blurring and feature loss during fast motion or rotation of the mobile robots with visual sensors occur so that the feature matching fails resulting in the decrease of accuracy and precision of system localiztion and mapping, a schemefusing the depth camera(RGB_D) with inertial measurement unit(IMU) is proposed in this paper, in which the ORB SLAM2 algorithm is used to estimate the pose, and the IMU information is used as a constraint to compensate for the lack of camera data. The measurement data of the two sensors are nonlinearly optimized by the loose coupling method based on extended Kalman filter. The experimental results of data set show that the method can effectively improve the positioning accuracy and system mapping effect of the robot.