The Pancharatnam phase of a Tavis-Cummings model with the coherent state in Kerr medium is investigated theoretically and numerically. An exact expression of the Pancharatnam phase under this general system is obtained. The numerical results show that the dipole-dipole coupling strength between two atoms can change the quasi-period of the Pancharatnam phase evolution,and it is also proved that the Pancharatnam phase contains the information of atoms and the field. In the nonlinear interaction regime, the nonlinearity can cause the chaotic behavior of the Pancharatnam phase.