Metal-2D material-metal photodetectors is the most common type of 2D material photodetectors. Due to the simple structure and the ease of integration with other systems, metal-2D material-metal photodetectors have received the widest range of attentions and research interest. The self-driven mode of this type of photodetectors has very low dark current, and then it is regarded as a promising new route for high performance infrared detection. However, there are two bottleneck problems for self-driven metal-2D material-metal photodetectors: (1) photoresponse cancellation caused by antisymmetric 2D material-contact junctions, (2) low responsivity due to limited light absorption of 2D materials. The recent progress on the study of metal-2D material-metal photodetectors with asymmetrically integrated plasmonic nanostructures was introduced, where asymmetrical light coupling was utilized to break the anti-symmetry between the photocurrents at the two contact-2D material junctions for self-driven net photoresponse, and the induced strong local field was utilized to enhance the absorptance and the responsivity of the 2D material. In the hybrid device of graphene and plasmonic nanocavities, the contrast between photoresponses at the two contacts is more than 100 times, which breaks through the problem of photoresponse cancellation caused by symmetric optical coupling. Due to the superior capability to couple the incident light into a localized mode, the plasmonic nanocavity can enhance the responsivity of graphene over one order of magnitude higher than a subwavelength metal grating.