Achieving high-accuracy localization in urban environments is challenging in autonomous driving. The existing LiDAR-based localization algorithms can ensure high accuracy in most cases; however, the localization problems in complex dynamic city scenes still need to be addressed. This study proposes a novel probabilistic localization framework to mitigate the accuracy degradation of the global positioning system caused by occlusion and to reduce the effective point cloud features caused by moving objects and changing environments in such scenarios. The proposed framework combines the improved multi-layer random sample consensus algorithm and the histogram filtering algorithm with the kernel density estimation method; this combination effectively overcomes the localization fluctuation of multi-layer random sample consensus in some scenes as well as the inefficiency and local optimum of histogram filtering when the pose error is large. The experimental results indicate that the proposed framework can provide more stable and accurate localization as well as tolerate larger initial pose errors compared with the existing localization methods when applied to complex dynamic city scenes.