In the field of Terracotta Warriors protection, to reduce the challenge of matching and splicing fragments of the Terracotta Warriors, more computer-aided technology is applied to the core link’s debris classification in the restoration of the broken Terracotta Warriors. The classification accuracy is low because of insufficient characteristic extraction of traditional Terracotta Warriors debris classification approaches and increased difficulty associated with data collection. In this paper, a depth classification model of Terracotta Warriors fragments based on data enhancement is presented. First, the existing dataset of Terracotta Warriors fragments was improved using conditional generative adversarial nets to achieve the dataset’s expansion of Terracotta Warriors. Second, the deep convolutional neural network was employed to automatically and effectively extract the debris feature information and achieve an effective debris classification effect. Third, the double-channel attention mechanism of the convolutional block attention module (CBAM) and the CutMix enhancement strategy were effectively introduced to significantly improve the deep classification model’s performance. Results on the experimental dataset of the Terracotta Warriors reveal that the presented approach is more accurate than the traditional classical debris classification approaches based on geometric, scale-invariant feature transform, and shape features as well as the multifeature fusion. It can effectively reduce the subsequent restoration work’s complexity, such as matching and stitching, and therefore improve the overall efficiency of the Terracotta Warriors’ restoration work.