• Advanced Photonics
  • Vol. 6, Issue 5, 056002 (2024)
Blake Wilson1,2,†, Yuheng Chen1,2, Daksh Kumar Singh1,2, Rohan Ojha1..., Jaxon Pottle3, Michael Bezick4, Alexandra Boltasseva1,2, Vladimir M. Shalaev1,2 and Alexander V. Kildishev1,*|Show fewer author(s)
Author Affiliations
  • 1Purdue University, Elmore Family School of Electrical and Computer Engineering, West Lafayette, Indiana, United States
  • 2Quantum Science Center, Oak Ridge National Laboratory, Oak Ridge, Tennessee, United States
  • 3Purdue University, School of Aeronautics and Astronautics, West Lafayette, Indiana, United States
  • 4Purdue University, Department of Computer Science, West Lafayette, Indiana, United States
  • show less
    DOI: 10.1117/1.AP.6.5.056002 Cite this Article Set citation alerts
    Blake Wilson, Yuheng Chen, Daksh Kumar Singh, Rohan Ojha, Jaxon Pottle, Michael Bezick, Alexandra Boltasseva, Vladimir M. Shalaev, Alexander V. Kildishev, "Authentication through residual attention-based processing of tampered optical responses," Adv. Photon. 6, 056002 (2024) Copy Citation Text show less
    PUF sampling process. An overview of the PUF tamper detection method using distance matrices of randomly positioned gold nanoparticles. The process consists of four primary stages. (i) Gold nanoparticles are randomly introduced, serving as a distinct physical system. (ii) The nanoparticles’ distance matrix is recorded and archived in a reference database. (iii) The system may experience external tampering or natural degradation that can modify its initial state. (iv) The distance matrix is reassessed and cross-referenced with the initial database to identify any potential tampering or other changes.
    Fig. 1. PUF sampling process. An overview of the PUF tamper detection method using distance matrices of randomly positioned gold nanoparticles. The process consists of four primary stages. (i) Gold nanoparticles are randomly introduced, serving as a distinct physical system. (ii) The nanoparticles’ distance matrix is recorded and archived in a reference database. (iii) The system may experience external tampering or natural degradation that can modify its initial state. (iv) The distance matrix is reassessed and cross-referenced with the initial database to identify any potential tampering or other changes.
    Distance matrix extraction from dark-field images. Nanoparticle dark-field images of size 448×448 pixels are prepared using dark-field microscopy. Then, the segmentation process classifies pixels as belonging to either a nanoparticle pattern or the dark-field background. Next, nanoparticle pattern pixel regions are clustered into local particle patterns, and their centers of mass (purple points) are extracted. Finally, the distance matrix is generated by evaluating all pairwise distances between these nanoparticle patterns. We visualize the distance matrix using its minimum spanning tree, despite the full tree being all-to-all. All scale bars represent 20 μm.
    Fig. 2. Distance matrix extraction from dark-field images. Nanoparticle dark-field images of size 448×448  pixels are prepared using dark-field microscopy. Then, the segmentation process classifies pixels as belonging to either a nanoparticle pattern or the dark-field background. Next, nanoparticle pattern pixel regions are clustered into local particle patterns, and their centers of mass (purple points) are extracted. Finally, the distance matrix is generated by evaluating all pairwise distances between these nanoparticle patterns. We visualize the distance matrix using its minimum spanning tree, despite the full tree being all-to-all. All scale bars represent 20  μm.
    Machine-learning-assisted authentication is trained by classifying synthetic posttamper measurements as being either adversarially tampered or naturally degraded, indicated by β^. We use a pretrained segmentation model, along with a labeled clustering algorithm, to compute the distance matrix and radii of the nanoparticles for both samples. Then, the discriminator network is trained by randomly choosing a synthetic tampering type according to the tampering Bernoulli distribution β∼B.
    Fig. 3. Machine-learning-assisted authentication is trained by classifying synthetic posttamper measurements as being either adversarially tampered or naturally degraded, indicated by β^. We use a pretrained segmentation model, along with a labeled clustering algorithm, to compute the distance matrix and radii of the nanoparticles for both samples. Then, the discriminator network is trained by randomly choosing a synthetic tampering type according to the tampering Bernoulli distribution βB.
    Adversarial tampering is introduced through tearing of the substrate, thereby separating the gold nanoparticles according to their distance from the tear line, and filling the tear with new nanoparticles uniformly distributed in the tear to match the original distribution. The tearing of the substrate is modeled as a random cut that shifts the nanoparticles based on the inverse square root of the perpendicular distance to the cut. (a), (b) The tearing coefficients w=0.01 and w=0.05 demonstrate the increased separation dependent on the tearing coefficient. (c) The normalized expected distance between nanoparticles is plotted for natural degradation, adversarial tearing without filling, and adversarial tearing with filling.
    Fig. 4. Adversarial tampering is introduced through tearing of the substrate, thereby separating the gold nanoparticles according to their distance from the tear line, and filling the tear with new nanoparticles uniformly distributed in the tear to match the original distribution. The tearing of the substrate is modeled as a random cut that shifts the nanoparticles based on the inverse square root of the perpendicular distance to the cut. (a), (b) The tearing coefficients w=0.01 and w=0.05 demonstrate the increased separation dependent on the tearing coefficient. (c) The normalized expected distance between nanoparticles is plotted for natural degradation, adversarial tearing without filling, and adversarial tearing with filling.
    RAPTOR uses an attention mechanism for prioritizing nanoparticle correlations across pretamper and posttamper samples before passing them into a residual, attention-based deep convolutional classifier. (a) RAPTOR takes the top 56 nanoparticles in descending order of radii to construct the distance matrices D and D′ and radii ρ and ρ′ from the pretamper and posttamper samples. (b) The radii and distance matrices form the query and value embeddings of an attention mechanism. The attention mechanism is then used alongside the raw distance matrices D′ and D, the soft weight matrix, and L2 matrix generated from the radii vectors for the classifier. (c) The classifier uses GELU activation and attention layers before applying a kernel layer and max pool layer. Then, the output is flattened into a multilayer perceptron to compute the final classification β^.
    Fig. 5. RAPTOR uses an attention mechanism for prioritizing nanoparticle correlations across pretamper and posttamper samples before passing them into a residual, attention-based deep convolutional classifier. (a) RAPTOR takes the top 56 nanoparticles in descending order of radii to construct the distance matrices D and D and radii ρ and ρ from the pretamper and posttamper samples. (b) The radii and distance matrices form the query and value embeddings of an attention mechanism. The attention mechanism is then used alongside the raw distance matrices D and D, the soft weight matrix, and L2 matrix generated from the radii vectors for the classifier. (c) The classifier uses GELU activation and attention layers before applying a kernel layer and max pool layer. Then, the output is flattened into a multilayer perceptron to compute the final classification β^.
    TaskMethodAverage Accuracy (%)Computation Time
    Distance matrix extractionSTEGO100% (ground truth)24 min for 1000 images
    ResNet attention CNN103 (BCE) or 99%27 ms for 1000 images
    Gaussian blur0.56 (BCE) or 99%33 ms for 1000 images
    DiscriminationRAPTOR97.6%80 ms for 1000 matrices
    AHD91.2%13.5 ms for 1000 matrices
    Hausdorff54.9%22.9 ms for 1000 matrices
    Procrustes58.2%3.30 s for 1000 matrices
    Table 1. Overall performance comparison of each method for distance matrix extraction and discrimination tasks. For all results in the table, a 1000-sample tensor was loaded onto an NVIDIA T4 GPU (except Procrustes, which used all CPU RAM) and batched at maximum capacity for the particular model. Accuracy is measured by the number of correct pixels or authentication classifications over the total. For semantic segmentation, we include the BCE loss to show a marginal advantage in using ResNet over Gaussian blur. The computation time is measured by preloading all data onto an NVIDIA T4 GPU or CPU RAM before recording the start time.
    Blake Wilson, Yuheng Chen, Daksh Kumar Singh, Rohan Ojha, Jaxon Pottle, Michael Bezick, Alexandra Boltasseva, Vladimir M. Shalaev, Alexander V. Kildishev, "Authentication through residual attention-based processing of tampered optical responses," Adv. Photon. 6, 056002 (2024)
    Download Citation