• Acta Optica Sinica
  • Vol. 39, Issue 5, 0515001 (2019)
Jun Meng* and Ximeng Zhao
Author Affiliations
  • College of Electrical Engineering, Zhejiang University, Hangzhou, Zhejiang 310027, China
  • show less
    DOI: 10.3788/AOS201939.0515001 Cite this Article Set citation alerts
    Jun Meng, Ximeng Zhao. Human Body Recognition and Positioning with Multiple Cameras Based on “Vibration Signals” from Skin Surfaces[J]. Acta Optica Sinica, 2019, 39(5): 0515001 Copy Citation Text show less
    Illustration of multiple people recognition and positioning with multiple cameras based on “vibration signals” from skin surfaces
    Fig. 1. Illustration of multiple people recognition and positioning with multiple cameras based on “vibration signals” from skin surfaces
    Principle derivation of difference and synchronization of “vibration signals” from skin surfaces
    Fig. 2. Principle derivation of difference and synchronization of “vibration signals” from skin surfaces
    Validation of synchronization and difference. (a) Experimental scene 1; (b) experimental scene 2; (c) example of IPPG signal acquisition for different skin areas of the same object under the same camera; (d) IPPG signal acquisition for different skin areas of different objects under the same camera
    Fig. 3. Validation of synchronization and difference. (a) Experimental scene 1; (b) experimental scene 2; (c) example of IPPG signal acquisition for different skin areas of the same object under the same camera; (d) IPPG signal acquisition for different skin areas of different objects under the same camera
    IPPG signal acquisition for the same skin area of different objects under multiple cameras
    Fig. 4. IPPG signal acquisition for the same skin area of different objects under multiple cameras
    Flow chart of human body recognition and positioning system based on multiple cameras
    Fig. 5. Flow chart of human body recognition and positioning system based on multiple cameras
    Acquisition of “vibration signals” from human skin surfaces
    Fig. 6. Acquisition of “vibration signals” from human skin surfaces
    Positioning with three cameras. (a) Top view of experimental scene; (b) principle diagram of positioning
    Fig. 7. Positioning with three cameras. (a) Top view of experimental scene; (b) principle diagram of positioning
    One frame of acquired video with three cameras and tracked skin areas
    Fig. 8. One frame of acquired video with three cameras and tracked skin areas
    IPPG signals acquired by camera 3
    Fig. 9. IPPG signals acquired by camera 3
    IPPG signals acquired by cameras 2 and 3
    Fig. 10. IPPG signals acquired by cameras 2 and 3
    Multiple coordinate calculation results of objects A and B
    Fig. 11. Multiple coordinate calculation results of objects A and B
    Experimental scene of 5 objects for recognition and positioning with 3 cameras. (a) Experimental scene for multi-object recognition and positioning; (b) top view of scene; (c) one frame of acquired video by three cameras and skin areas
    Fig. 12. Experimental scene of 5 objects for recognition and positioning with 3 cameras. (a) Experimental scene for multi-object recognition and positioning; (b) top view of scene; (c) one frame of acquired video by three cameras and skin areas
    Schematic of accuracy area distribution of camera
    Fig. 13. Schematic of accuracy area distribution of camera
    Coordinate calculation results of 5 objects
    Fig. 14. Coordinate calculation results of 5 objects
    Method for people recognition and positioningBased on face recognitionBased on “vibration signals” from skin surfaces
    Matching featureImage feature (geometric feature)“Vibration signal” from skin surfaces (light intensity change of skin surfaces caused by blood volume change over time)
    Area of interestFace (local area)Bare skin (whole body area)
    Requirement for areas of interestLight, facial expression, posture, angle, occlusion, image resolutionLight
    SafetyFacial features easy to copy or change“Vibration signal” from skin surfaces related to heartbeat and not easily changed
    DatabaseNeed for databaseNo need for database, instant matching
    Data to transferImageOne-dimensional time series
    Table 1. Method comparison for multiple people recognition and positioning with multiple cameras
    Cosine after detrendA-right armA-left armB-right armB-left armA-headB-head
    A-right arm1.00000.6797-0.13380.14210.62150.0128
    A-left arm1.00000.04670.14220.66420.0084
    B-right arm1.0000-0.7226-0.1806-0.4908
    B-left arm1.00000.20610.6557
    A-head1.00000.0102
    B-head1.0000
    Table 2. Similarity of IPPG signals acquired by camera 3
    Cosine after detrendCam2-A-left armCam2-B-left armCam2-A-right armCam2-B-headCam2-A-head
    Cam3-A-right arm0.493862-0.331650-0.4242300.231214-0.40585
    Cam3-A-left arm0.581386-0.227610-0.7547500.242505-0.67486
    Cam3-B-right arm-0.1395600.3371640.180864-0.394820-0.02642
    Cam3-B-left arm0.349941-0.403400-0.4498300.406926-0.26560
    Cam3-A-head0.537608-0.180590-0.7038500.242280-0.43896
    Cam3-B-head0.085119-0.468560-0.2323200.436905-0.17860
    Table 3. Similarity of IPPG signals acquired by cameras 2 and 3
    SampleSkin area [format: object number-skin number (camera number)]
    Cluster 12-1(Cam1),2-2(Cam1), 4-1(Cam1), 2-2(Cam2), 2-2(Cam3), 2-1(Cam3), 1-2(Cam2), 4-2(Cam2), 1-1(Cam1), 4-1(Cam2)
    Cluster 23-1(Cam2), 1-1(Cam3), 1-2(Cam1)
    Cluster 33-2(Cam1), 1-1(Cam2), 3-2(Cam2), 3-2(Cam3), 3-1(Cam1), 3-1(Cam3)
    Cluster 44-1(Cam3), 4-2(Cam3)
    Cluster 55-1(Cam2), 5-2(Cam2), 5-1(Cam3), 5-1(Cam1)
    Table 4. Classification results of similarity comparison and normalization
    ClassPrecisionRecall rate
    Class 10.670.40
    Class 20.501.00
    Class 30.830.83
    Class 41.000.40
    Class 51.001.00
    Average0.800.73
    Table 5. Precision and recall rates of classification results of similarity comparison and normalization
    Jun Meng, Ximeng Zhao. Human Body Recognition and Positioning with Multiple Cameras Based on “Vibration Signals” from Skin Surfaces[J]. Acta Optica Sinica, 2019, 39(5): 0515001
    Download Citation