• Laser & Optoelectronics Progress
  • Vol. 62, Issue 9, 0922002 (2025)
Jiaxi Duan1,2,**, Dewen Cheng1,2,*, Lianbing Wang2, Shuxian Sun2..., Qichao Hou2,3, Yang Wang2, Pengli Wang2, Xin Che2, Weihong Hou2,3 and Donghua Wang3|Show fewer author(s)
Author Affiliations
  • 1Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
  • 2Ned Co., Ltd., Beijing 100081, China
  • 3Dega Smart Photoelectric Technology Co., Ltd.,Zhenjiang 212132, Jiangsu , China
  • show less
    DOI: 10.3788/LOP241652 Cite this Article Set citation alerts
    Jiaxi Duan, Dewen Cheng, Lianbing Wang, Shuxian Sun, Qichao Hou, Yang Wang, Pengli Wang, Xin Che, Weihong Hou, Donghua Wang. Design of a Split Optical See-Through Head-Mounted Display System with Integrated Multispectral Sensors[J]. Laser & Optoelectronics Progress, 2025, 62(9): 0922002 Copy Citation Text show less
    Functional architecture diagram of OST-HMD system with integrated multi-spectral sensors
    Fig. 1. Functional architecture diagram of OST-HMD system with integrated multi-spectral sensors
    Schematic diagram of split OST-HMD
    Fig. 2. Schematic diagram of split OST-HMD
    Schematic diagram of signal transmission architecture for head-mounted display module
    Fig. 3. Schematic diagram of signal transmission architecture for head-mounted display module
    Frameflow of integrated multi-spectral sensors signal acquisition system
    Fig. 4. Frameflow of integrated multi-spectral sensors signal acquisition system
    Layout of OST display module system
    Fig. 5. Layout of OST display module system
    Components and prototype of the optical display module. (a) All components of the display system; (b) overall appearance of the display module
    Fig. 6. Components and prototype of the optical display module. (a) All components of the display system; (b) overall appearance of the display module
    Physical picture of the low-light camera module
    Fig. 7. Physical picture of the low-light camera module
    Physical picture of the infrared light camera module
    Fig. 8. Physical picture of the infrared light camera module
    Physical picture of the visible light camera module
    Fig. 9. Physical picture of the visible light camera module
    Self-developed driver board circuit diagram
    Fig. 10. Self-developed driver board circuit diagram
    Binocular display driver board. (a) Front layout of the driver board; (b) back layout of the driver board
    Fig. 11. Binocular display driver board. (a) Front layout of the driver board; (b) back layout of the driver board
    Handheld prototype structure of integrated multispectral signal acquisition module and freeform prism display module
    Fig. 12. Handheld prototype structure of integrated multispectral signal acquisition module and freeform prism display module
    The prototype integrating multi-spectral signal acquisition module and display module. (a) Whole structure prototype with handheld structure and movable bracket; (b) head-mounted structure integrating multi-spectral signal acquisition module and display module
    Fig. 13. The prototype integrating multi-spectral signal acquisition module and display module. (a) Whole structure prototype with handheld structure and movable bracket; (b) head-mounted structure integrating multi-spectral signal acquisition module and display module
    Embedded core computing platform
    Fig. 14. Embedded core computing platform
    The display effect of the optical display module. (a) Input image displayed in the MicroOLED; (b) output image taken by the camera at the system output pupil
    Fig. 15. The display effect of the optical display module. (a) Input image displayed in the MicroOLED; (b) output image taken by the camera at the system output pupil
    Fusion of a virtual cup with a real cup. (a) Real cup; (b) fusion display of virtual cup and real cup
    Fig. 16. Fusion of a virtual cup with a real cup. (a) Real cup; (b) fusion display of virtual cup and real cup
    Images captured by different sensors. (a) Image captured by low-light module; (b) image captured by infrared module; (c) image captured by visible light module
    Fig. 17. Images captured by different sensors. (a) Image captured by low-light module; (b) image captured by infrared module; (c) image captured by visible light module
    ParameterExplanation
    CPU• Architecture: Dual Cortex-A72 + Quad Cortex-A53, 64-bit CPU
    • Frequency is up to 1.8 GHz
    GPU• Mali-T860 GPU, OpenGL ES1.1/2.0/3.0/3.1, OpenCL
    • Supports AFBC (ARM frame buffer compression)
    Memory• Dual channel DDR3-1866/DDR3L-1866/LPDDR3-1866/LPDDR4
    • Support eMMC 5.1 with HS400, SDIO 3.0 with HS200
    Multi-media• 4K VP9 and 4K 10 bit H265/H264 video decoders, up to 60 frame/s
    • 1080P other video decoders (VC-1, MPEG-1/2/4, VP8)
    • 1080P video encoders for H.264 and VP8
    • Video post processor: de-interlace, de-noise, enhancement for edge/detail/color
    Display• Dual VOP: one supports 4096×2160 with AFBC supported; the other supports 2560×1600
    • Dual channel MIPI-DSI (4 lanes per channel)
    • eDP 1.3 (4 lanes with 10.8 Gb/s) to support display, with PSR
    • HDMI 2.0a for 4K 60 Hz with HDCP 1.4/2.2
    • DisplayPort 1.2 (4 lanes, up to 4K 60 Hz)
    • Supports Rec.2020 and conversion to Rec.709
    Interface• Dual 13M ISP and dual channel MIPI CSI-2 receive interface
    • Dual USB 3.0 with type-C supported
    • PCIe 2.1 (4 full-duplex lanes )
    • Embedded low power MCU for other applications
    • 8 channels I2S supports 8 channels RX or 8 channels TX
    Package• FCBGA828 21mm×21mm, 0.65mm pitch
    Table 1. RK3399 chip parameter table
    DatasetParameter
    DetectorHigh sensitivity camera core
    Detector size1 inch
    Resolution1024×768
    Frame rate25 Hz
    Pixel size13 μm
    Weight(include lens)35.13 g
    Length size≤38 mm
    Table 2. Low-light camera module specification sheet
    DatasetParameter
    DetectorUncooled micro thermal module
    Resolution640×512, 384×288
    Pixel size12 μm
    Frame rate25 Hz, 50 Hz
    Weight(include lens)≤26.4 g
    Length size≤37 mm
    Table 3. Infrared light camera module specification sheet
    Jiaxi Duan, Dewen Cheng, Lianbing Wang, Shuxian Sun, Qichao Hou, Yang Wang, Pengli Wang, Xin Che, Weihong Hou, Donghua Wang. Design of a Split Optical See-Through Head-Mounted Display System with Integrated Multispectral Sensors[J]. Laser & Optoelectronics Progress, 2025, 62(9): 0922002
    Download Citation