Herein, a method for extracting the Jones matrix of a material in the terahertz (THz) wave band is proposed. A THz focal-plane imaging system was utilized to coherently measure the longitudinal component of a converging THz field and to determine the THz polarization. Furthermore, the Jones matrix of the material was extracted by measuring the polarization of a reference THz field and the polarizations of the THz fields transmitted from a sample in its original position and with a 90° rotation. The proposed method was employed to measure the Jones matrices of a THz polarizer and a wave plate with different azimuthal angles, and the experimental results were consistent with theoretical calculations. This demonstrated the feasibility of the proposed method to characterize the anisotropy of materials in the THz frequency range.