This study aims to address the real-time limitations of image matching and the problem of ghosting in image projection stitching. Hence, a fast and improved scale invariant feature transform (SIFT) image stitching and ghosting optimization algorithm was proposed. First, feature points were classified based on the similarity of the shared information between the images, and then, the SIFT algorithm was used to detect and extract the feature points of similar coincident regions. This approach required the algorithm to spend less time on the useless regions. At the image stitching stage, the projection matrix was calculated by feature points, and rough projection was performed. Thereafter, according to the density of the area where the feature points were located, secondary projection splicing was performed on the dense feature points area by optimal fitting transformation to reduce the ghosting problem. Experiments are performed, and the results demonstrate that compared with the traditional SIFT algorithm, the efficiency of feature point extraction is improved by approximately 58%. Similarly, the comparison by an objective evaluation index show that image stitching improved by approximately 10%.