Single-pixel imaging (SPI) can capture 2D images of the target with only a nonpixelated detector, showing promising application potential in nonvisible spectral imaging, low-photon imaging, lidar, and other extreme imaging fields. However, the imaging mechanism of traditional SPI makes it difficult to achieve high imaging speed, which is a primary barrier for its widespread application. To address this issue, in this work, we propose and demonstrate a novel high-speed 2D and 3D imaging scheme based on traditional SPI, termed time-resolved single-pixel imaging (TRSPI). Previous SPI works mainly utilize correlation between a stable target and iterative illumination masks to reconstruct a single image. In TRSPI, by further exploiting correlation information between a dynamic scene and every static mask, we can reconstruct a series of time-varying images of the dynamic scene, given the dynamic scene is repetitive or reproducible. Experimentally, we conducted 2D and 3D imaging on a rotating chopper with a speed of 4800 revolutions per minute (rpm), and imaging speeds up to 2,000,000 fps. It is believed that this technology not only opens up a novel application direction for SPI, but also will provide a powerful solution for high-speed imaging.