Understanding inter-site mutual mode interaction in coupled physical systems is essential to comprehend large compound systems, as this local interaction determines the successive multiple inter-site energy transfer efficiencies. In the present study, we demonstrate that only the non-Hermitian coupling can correctly account for the light transfer between two coupled optical cavities. We also reveal that the non-Hermitian coupling effect becomes crucial as the system dimension decreases. Our results provide important insight for handling general-coupled devices in the subwavelength regime.