Vector Tracking Loop (VTL) is one of the most promising signal tracking technologies of Global Navigation Satellite System (GNSS) receiver technologies. In the traditional GNSS receiver, Phase Lock Loop (PLL) is employed to complete carrier signal tracking, and then Least Squares Method (LSM) or Kalman Filter (KF) is utilized to estimate navigation information. In contrast, VTL utilizes a KF to complete signal tracking and navigation estimation, which has better signal tracking abilities. In order to improve VTL positioning accuracy, Graph Optimization (GO) based VTL is proposed in this paper. In this method, GO is used to replace KF. Compared with KF, GO regards the state and measurement equations of VTL filter as constraints, whose optimal solution is obtained by Levenberg-Marquardt algorithm, so that the optimal navigation estimation is obtained. Field tests show that the standard deviation of 3D position errors is reduced respectively by 40.5%, 50.9% and 41.7%, and the mean values of 3D position errors are decreased respectively by 43.0%, 54.9% and 34.7%.