In this paper, we propose and demonstrate a high-performance mercury ion sensor with sub-nM detection limit, high selectivity, and strong practicability based on the small molecule of the 4-mercaptopyridine (4-MPY) modified tilted fiber Bragg grating surface plasmon resonance (TFBG-SPR) sensing platform. The TFBG-SPR sensor has a rich mode field distribution and a narrow bandwidth, which can detect the microscopic physical and chemical reactions on the sensor surface with high sensitivity without being disturbed by the external temperature. For the environmental compatibility and highly efficient capture of the toxic mercury ion, 4-MPY is modified on the sensor surface forming a stable (4-MPY)-Hg-(4-MPY) structure due to the specific combination between the nitrogen of the pyridine moiety and the Hg²⁺ via multidentate N-bonding. Moreover, gold nanoparticles (AuNPs) are connected to the sensor surface through the (4-MPY)-Hg-(4-MPY) structure, which could play an important role for signal amplification. Under the optimized conditions, the limit of detection of the sensor for mercury ions detection in the solution is as low as 1.643×10^–10 M (0.1643 nM), and the detection range is 1×10^–9 M – 1×10^–5 M. At the same time, the mercury ion spiked detection with tap water shows that the sensor has the good selectivity and reliability in actual water samples. We develop a valuable sensing technology for on-time environmental Hg²⁺ detection and in-vivo point of care testing in clinic applications.