Land fragmentation is widely known to have an impact on farm performance. However, previous studies investigating this impact mainly focused on a single crop, and only limited data from China are available. This study considers multiple crops to identify the impact of land fragmentation (LF), as well as cropping system (CS), on farm productivity and the efficiency of grain producers in the North China Plain (NCP), using Cangxian County of Hebei Province as an example. Detailed household- and plot-level survey data are applied and four stochastic frontier and inefficiency models are developed. These models include different sets of key variables in either the production function or the inefficiency models, in order to investigate all possibilities of their influences on farm productivity and efficiency. The results show that LF plays a significant and detrimental role, affecting both productivity and efficiency. A positive effect is evident with respect to the CS variable, i.e., multiple cropping index (MCI), and the wheat-maize double CS, rather than the maize single CS, is usually associated with higher farm productivity and efficiency. In addition to LF and CS, four basic production input variables (labor, seed, pesticide and irrigation), also significantly affect farmers’ productivity, while the age of the household head and the ratio of the off-farm labor to total labor are significantly relevant to technical inefficiency. Policies geared toward the promotion of land transfer and the rational adjustment of cropping systems are recommended for boosting farm productivity and efficiency, and thus maintaining the food supply while mitigating the overexploitation of groundwater in the NCP.