Graphene has some unique properties, such as ultra-high carrier mobility, zero band gap, broadband response, which make it a promising material in infrared photodetection. In this review, the development history of graphene-based infrared detectors was analyzed, and the mechanism of relevant photoelectric response was summarized. The responsivity, wave-band, response speed and device structure were sorted out. The challenges of material preparation and process compatibility of graphene-based detectors were also discussed and prospected.