Machine learning has become an important transformative method to explore novel materials, but the small sample size and high noise of material data bring a great challenge to data-driven research and development. To address the challenge, unsupervised learning was applied to discover perovskite materials with a high dielectric constant. Twenty perovskite materials with a high dielectric constant (i.e., BaHfO3 and BiFeO3) were screened out via iterative clustering. We performed dimensionality reduction analysis and descriptors analysis including elements, crystal structure and tolerance factors to find the underlying trend and the relationship between ABO3 structure and dielectric constant. This method can provide an idea for solving the lack of material data labels, which can be also applied to screen other novel functional materials.