• Photonics Research
  • Vol. 10, Issue 10, 2434 (2022)
Navid Bani Hassan1、†, Fahimeh Dehkhoda2、†, Enyuan Xie1, Johannes Herrnsdorf1、*, Michael J. Strain1, Robert Henderson2, and Martin D. Dawson1
Author Affiliations
  • 1Institute of Photonics, Department of Physics, University of Strathclyde, Technology and Innovation Centre, Glasgow G1 1RD, UK
  • 2School of Engineering, Institute for Integrated Micro and Nano Systems, University of Edinburgh, Edinburgh EH9 3JL, UK
  • show less
    DOI: 10.1364/PRJ.455574 Cite this Article Set citation alerts
    Navid Bani Hassan, Fahimeh Dehkhoda, Enyuan Xie, Johannes Herrnsdorf, Michael J. Strain, Robert Henderson, Martin D. Dawson. Ultrahigh frame rate digital light projector using chip-scale LED-on-CMOS technology[J]. Photonics Research, 2022, 10(10): 2434 Copy Citation Text show less

    Abstract

    Digital light projector systems are crucial components in applications, including computational imaging, fluorescence microscopy, and highly parallel data communications. Current technology based on digital micromirror displays are limited to absolute frame rates in the few tens of kiloframes per second and require the use of external light sources and coupling optics. Furthermore, to realize gray-scale pixel values using duty cycle control, frame rates are reduced proportionally to the number of gray levels required. Here we present a self-emissive chip-scale projector system based on micro-LED pixels directly bonded to a smart pixel CMOS drive chip. The 128×128 pixel array can project binary patterns at up to 0.5 Mfps and toggle between two stored frames at megahertz rates. The projector has a 5-bit gray-scale resolution that can be updated at up to 83 kfps, and can be held in memory as a constant bias for the binary pattern projection. Additionally, the projector can be operated in a pulsed mode, with individual pixels emitting pulses down to a few nanoseconds in duration. Again, this mode can be used in conjunction with the high-speed spatial pattern projection. As a demonstration of the data throughput achievable with this system, we present an optical camera communications application, exhibiting data rates of >5 Gb/s.

    1. INTRODUCTION

    Digital light projection (DLP) systems are now a well-established technology, with advances driven by the demands of cinematic and consumer markets. Systems with kiloframes per second (kfps) update rates and pixel counts in the megapixel range are widely available, serving both consumer video and scientific applications. Commercial developments are strongly driven by demand for high pixel count devices for digital projection and displays, though a number of other application areas have emerged including optical wireless communications [1], data through display [25], visible light positioning [6], single-pixel and 3D imaging [713], optomechanical manipulation [14], fluorescence lifetime imaging [15], and optogenetic stimulation [16,17]. These applications benefit from, or even require, pixel modulation rates well beyond the typical frame rates of visual displays.

    Current commercially available DLP systems are mainly based on two forms of non-self-emissive technology, liquid crystal displays (LCDs) and digital micromirror displays (DMDs) [18,19]. LCDs use electronic control of nematic liquids, commonly sandwiched between polarization films, to control the absorption or phase change of light through a pixel [19]. DMDs are based on microelectromechanical systems (MEMS) devices, where the reflective surface of a pixel can be orientated to pass or block incident light through the projection path [18]. DMDs are particularly attractive due to their pixel switching times and compatibility with drive electronics. While commercial development has enabled the impressive resolution of these systems, their frame rate is limited by the physical motion required to switch pixels and the data-handling capacity required to update megapixels frames. Furthermore, gray-scale operation of DMD displays uses duty-cycle control of pixel switching [18], consequently reducing the maximum achievable frame rate of the device inversely with the number of gray levels required.

    In parallel to these developments, advances in semiconductor materials, and in particular the III-nitrides, have enabled the realization of high brightness LED pixel emitters with dimensions from the millimeter to micrometer range [20]. The spectral quality, high brightness, and potential for integration with electronic backplanes, have seen deployment of these devices in displays ranging in dimension from mobile phone interfaces to super-large screen installations for advertising or theatrical installations. In addition to their emissive properties, LED pixels have another key advantage to offer in DLP systems: namely, their high modulation bandwidth, with individual devices demonstrated at up to gigahertz rates [21], and the potential for short pulse emission in the subnanosecond regime [22].

    The planar format of LED pixel arrays allows them to be directly interfaced with electronic drive chips, for example, through flip-chip bonding [23]. Arrays of LEDs with individual pixel control electronics represent a new form of compact microdisplay/projector [20]. In previous work, we have reported on LED arrays with 16×16 and 10×40 pixel arrays, integrated onto complementary metal-oxide semiconductor (CMOS) driver chips [6,24,25], along with proof-of-concept demonstrations including gigabytes-per-second data communications [6,26,27], time-of-flight ranging [8], spatial multiplexing, and navigation [6]. These initial integrated chip systems were designed to allow basic investigations of multiple modes of operation and were therefore very flexible devices, but limited in total pixel number and effective frame rate, the latter being 30 kfps in the best case.

    In this work, we present a custom-designed chip-scale LED-on-CMOS DLP using a 128×128 pixel array optimized for high-speed spatiotemporal pattern projection. The device is designed with a versatile electronic interface to allow pattern projection at up to 0.5 Mfps, 5 ns-pulsed operation, gray-scale pixel intensity, and pattern toggling in excess of 2 MHz using on-chip memory. Furthermore, and importantly, these individual modes can be operated simultaneously, for example, allowing image projection at video rates, together with high-speed data transmission. There is, to our knowledge, no other technology that currently provides megaframes per second (Mfps) rates at >10,000 pixels, or that combines patterned projection at these rates with nanosecond pulsing capability and gray-scale intensity control.

    This paper presents the performance of the chip in each operation mode, in addition to an example application in high-speed optical camera communications (OCCs), where we demonstrate >5  Gb/s. This compares to what is, to our knowledge, the previous highest OCC data rate of 20 Mb/s [28,29]. The LED-on-CMOS system has been developed as a technology platform, and it offers significant advantages for the breadth of applications using DLP technology, including fluorescence lifetime imaging and machine vision systems.

    2. DLP HARDWARE

    The device presented here is an array of 128×128, 30-μm-diameter LED pixels at a 50-μm pitch, where each pixel is addressed through an individual driver at a maximum full resolution frame rate of 0.5 Mfps, and a pattern toggling rate of 2 MHz (8 MHz at compromised peak power and duty cycle). Each pixel has the capability to modulate its optical output with 4 ns long pulses. Further technical details are given in Appendix A.

    A. Device Fabrication

    The LED array was fabricated using a c-plane GaN-on-sapphire wafer with standard LED fabrication processes that follow the design guidelines in Ref. [25]. Figure 1 shows microscope images of the fabricated CMOS driver chip and LED array system.

    Photograph of the mounted LED projector on CMOS chip, co-packaged with an FPGA controller. The projector is here displaying a diagonal cross pattern. Magnified inset images show the mounted LED array on the CMOS chip and a zoomed view of the LED pixel contact pads, imaged through the sapphire substrate.

    Figure 1.Photograph of the mounted LED projector on CMOS chip, co-packaged with an FPGA controller. The projector is here displaying a diagonal cross pattern. Magnified inset images show the mounted LED array on the CMOS chip and a zoomed view of the LED pixel contact pads, imaged through the sapphire substrate.

    The initial prototypes reported here emit at 450 nm, though the entire wavelength range of the GaN material system (280  nm  to>520  nm) is accessible with this technology. The best device fabricated has an 86% pixel yield, i.e., >14,000 active pixels. This yield is representative of the research prototyping facilities employed and can be expected to be significantly improved in industrial standardized semiconductor fabrication facilities.

    B. Smart Pixel Driver Details and Functionality

    Each individual pixel driver on the chip comprises a 5-bit current digital-to-analog converter (DAC), a separate current source with 2 mA driving capability, a 2-bit in-pixel memory for high-speed pattern toggling, and control signals to enable different operation modes. Figure 2 shows a schematic circuit diagram of these elements and their connections. The chip infrastructure that supplies signals and power to each pixel is detailed in Appendix B. A simplified timing diagram is presented in Fig. 3.

    Simplified schematic of the driver circuit for one pixel, showing 5-bit DAC, in-pixel memory, and control signals.

    Figure 2.Simplified schematic of the driver circuit for one pixel, showing 5-bit DAC, in-pixel memory, and control signals.

    Simplified timing diagram of the digital control signals supplied to the CMOS by an FPGA.

    Figure 3.Simplified timing diagram of the digital control signals supplied to the CMOS by an FPGA.

    The optical output power of a single pixel driven by the DAC is shown in Fig. 4(a), and a good linear behavior is observed in the output power apart from a small kink between gray levels 15 and 16, which is a well-known effect in this type of CMOS current DAC [30]. An example of a gray-scale image [the project logo of the Engineering and Physical Sciences Research Council (EPSRC)-supported QuantIC program; see Funding and Acknowledgement sections] from the device is shown in Fig. 4(b). If a sequence of binary patterns is being displayed, the DAC can be used to either superimpose a gray-scale image or mitigate possible manufacturing variations in pixel brightness. It is also possible to display a sequence of true gray-scale patterns at up to 83 kfps, in which case predistortion or postequalization techniques can be used to mitigate pixel brightness variations.

    (a) Optical output power of a single pixel for different DAC settings; (b) gray-scale image from the LED projector chip using the DAC gray levels; (c) digital version of the image projected in (b).

    Figure 4.(a) Optical output power of a single pixel for different DAC settings; (b) gray-scale image from the LED projector chip using the DAC gray levels; (c) digital version of the image projected in (b).

    3. DEMONSTRATION OF SYSTEM OPERATION MODES

    There are four different modes of pattern switching accessible using the custom driver chip: rolling pixel updateglobal pixel updateglobal pixel togglingnanosecond pulse mode

    Mode 1 can be accessed for both binary and gray-scale pattern projection. Modes 2 and 3 are accessible for binary patterns. In all cases, gray-scale information can be prestored on-chip and the binary pattern projection, toggling or pulsing, can be applied on top of this pre-existing bias set point. Details of each operation mode are presented in the following sections.

    A. High-Speed Pattern Projection and OCC

    For binary pixel on–off state information, there are two update modes: rolling and global pixel update. For these cases, a single, full-frame, binary pattern can be loaded onto the chip within 2 μs, thus enabling 0.5 Mfps at full resolution. In the rolling pixel mode, each pixel is updated individually in sequence, with the value held until the next update. In the global pixel mode, the full frame is refreshed in one shot once all of the pattern data are loaded.

    In addition to the binary pixel on–off data, gray-scale values can be sent to each pixel in the array. Significantly, in contrast to DMD devices, the gray-scale value corresponds to the pixel drive current and is not a duty cycle effect that relies on multiple binary frames for implementation. The gray-scale pattern that is held by the 5-bit in-pixel DAC can be loaded within 12 μs, corresponding to 83 kfps. As detailed in Appendix B, input data can be selectively written to the binary pixel state or the DAC, corresponding to the switch control signal. Therefore, a gray-scale image can be stored on the chip and used as a set point for rapid pixel on–off modulation.

    To demonstrate the potential of this high-speed binary projection, we used it in an OCC application. Full details of the experimental setup and data handling are presented in Appendix C. The camera was sampling at 0.8 Mfps, and the projector operated at 0.25 and 0.4 Mfps, the latter corresponding to the camera’s Nyquist sampling limit. Figure 5 shows eye diagrams and detected symbol level histograms for 104 frame data sets captured for both operating rates.

    (a) Measured eye diagram for 104 transmitted frames at 0.25 Mfps rate and (b) associated signal level histogram; (c) measured eye diagram at 0.4 Mfps and (d) associated signal level histogram.

    Figure 5.(a) Measured eye diagram for 104 transmitted frames at 0.25 Mfps rate and (b) associated signal level histogram; (c) measured eye diagram at 0.4 Mfps and (d) associated signal level histogram.

    The eye diagram for the 0.25 Mfps case is very clear and represents error-free transmission (for the 104 sample limit). The signal level histogram shows clear separation of the 0 and 1 bit levels. For the 0.4 Mfps projector rate, the eye diagram is more closed than for the 0.25 Mfps case. This is due to the sampling limit of the camera rather than intensity-level variations in the projector, with clear time jitter measurement limitations present in the eye diagram. The symbol-level histogram shows very clear distinction between the 0 and 1 bit levels and again, error-free data transmission up to 104 transmitted frames was achieved. Taking into account the 86% pixel yield of the chip, aggregate data rates of 3.5 Gb/s and 5.6 Gb/s were transmitted for the 0.25 Mfps and 0.4 Mfps frame rates, respectively.

    B. High-Speed Pattern Toggling

    Using the 2-bit in-pixel memory function, it is possible to exceed the switching rate of the data projection modes detailed above in a two-pattern toggling mode. In this mode, two independent frames can be stored using the in-pixel memory and switched using a global clock signal. Toggling was first assessed using the same setup as for OCC. We selected two pattern sets for the projector system toggling tests: (1) checkerboard switching to all pixels off (C2B) and (2) two spatially complementary checkerboard (C2C) patterns. Figure 6 shows raw captured images, where each image shows a 20×2 pixel subsection of the LED array. Figures 6(a) and 6(b) are temporally adjacent image frames at 0.8 Mfps for the C2B patterns. Similarly, Figs. 6(c) and 6(d) are temporally adjacent frames of the C2C patterns. The LED array was toggling at 0.8 Mfps; these frames are representative ones from a video where the LED array and camera capture were in-phase with one another.

    Representative LED projection frames captured with a high-speed camera at 0.8 Mfps. Visible is the partial footprint of the last two columns of the LED chip: (a) frame #1 C2B, (b) frame #2 C2B, (c) frame #1 C2C, and (d) frame #2 C2C. (e) Measured photosignal from global array toggling at 2 Mfps.

    Figure 6.Representative LED projection frames captured with a high-speed camera at 0.8 Mfps. Visible is the partial footprint of the last two columns of the LED chip: (a) frame #1 C2B, (b) frame #2 C2B, (c) frame #1 C2C, and (d) frame #2 C2C. (e) Measured photosignal from global array toggling at 2 Mfps.

    To determine the pattern switching limits, the array was toggled between all pixels off and a pattern with all pixels on at the highest DAC setting of 31, and the optical output was measured with a photomultiplier tube (PMT). Figure 6(e) shows the recorded waveform at a toggling rate of 2 Mfps. Above 2 Mfps, a reduction in duty cycle and peak power was observed, and at 8 Mfps, the duty cycle had reduced to 25%. This behavior was independent of the number of pixels that were switched.

    C. Nanosecond Pulsed Mode

    Compared with other display technologies, the micro-LED platform has the unique capability of directly emitting nanosecond optical pulses [22]. In the device presented here, this capability is accessed through the 2-mA driver, which is controlled by a separate global control signal that can be switched on a nanosecond time scale. Pixels are enabled to respond to the global pulse signal superimposed with the binary pattern, thus allowing spatial patterns to be pulsed, where the patterns can be updated at the same rate, as detailed previously. The nanosecond pulsing function was assessed by loading binary patterns with different numbers and distributions of active pixels, and pulsing them with electronic pulse widths between 5 and 100 ns at repetition rates between 10 and 100 MHz. The optical pulses were resolved using a PMT connected to an oscilloscope with 1 GHz bandwidth, and the spatial patterns were verified using a slow-response optical camera.

    Figure 7(a) shows a representative spatial projection pattern that is being driven in nanosecond pulsed mode. Representative temporal waveforms at 10 MHz repetition rate, 5 ns pulse duration, and with the number of active pixels as a parameter, are shown in Fig. 7(b), highlighting the consistent pulse shape over a wide range of active device numbers. The number of active pixels is limited to 5000 in these measurements, since optical emission was significantly reduced beyond this, as detailed below. The electrical drive pulse width is 5 ns, while the emitted optical pulse width is 4  ns due to the rise and fall times of the LED pixel. As shown in Fig. 7(c), the measured optical pulse width reduces as a function of increasing active pixel count. The effect is most pronounced for the 10 MHz repetition rate measurements, where the pulse width is dominated by the rise and fall times of the LED. The capacitive loading associated with increasing numbers of active pixels leads to a delay in the optical rise time and thus to pulse shortening. At repetition rates above 50 MHz the full width at half-maximum (FWHM) pulse width is more stable, but is attended by a reduction in extinction ratio. This reduced pulse amplitude at duty cycles greater than 0.25 reduces the impact of the capacitive loading and therefore leads to the more stable pulse widths as a function of active pixel numbers as shown. The relationship between electrical drive pulse and optical pulse widths is shown in Fig. 7(d).

    (a) Optical micrograph of a pattern displayed in nanosecond pulsed mode; (b) optical waveform of 5 ns long pulses as a function of number of pulsing pixels; (c) optical pulse duration as a function of the number of pixels for a 5 ns input pulse; (d) optical pulse duration as a function of electrical input pulse duration; (e) relative delay of the pulses emitted by individual pixels within one row; (f) relative delay of the pulses emitted by individual pixels within one column.

    Figure 7.(a) Optical micrograph of a pattern displayed in nanosecond pulsed mode; (b) optical waveform of 5 ns long pulses as a function of number of pulsing pixels; (c) optical pulse duration as a function of the number of pixels for a 5 ns input pulse; (d) optical pulse duration as a function of electrical input pulse duration; (e) relative delay of the pulses emitted by individual pixels within one row; (f) relative delay of the pulses emitted by individual pixels within one column.

    To assess the effect of the distribution of the global pulse signal in the nanosecond time regime, the timing delay of emitted pulses across the array was measured. Local arrays of the LED chip were imaged onto the PMT, and the received signal was compared to the pulse trigger signal to measure relative delay. Figures 7(e) and 7(f) show the relative time delay between optical pulses emitted by individual pixels at various locations on the chip, arranged by column or row, respectively. The time delay between columns on-chip does not show a clear trend, with a scatter of <0.1  ns. There is, however, a clear time delay trend with respect to the chip rows, which is related to the signal distribution of the CMOS driver chip. The maximum delay scatter across the full chip is still <0.7  ns, significantly lower than the pulse width. If an even lower time delay scatter were required, then this could be addressed by a dedicated redesign of the signal distribution network in the driver chip.

    The emitted optical pulse energy depends on the number of active pixels, as shown in Fig. 8, where the pulse duration was 5 ns and the repetition rate was 10 MHz. From Fig. 8(a), the total aggregated pulse energy increases as a function of active pixel number, then goes through a turning point, reducing as further pixels are added, beyond 1000 active pixels. At the maximum emission energy, the average chip current is 0.07 A, corresponding to a peak switching current of 1.5 A. As the number of active pixels increases, the average and peak currents increase to 0.12 and 2.4 A, respectively. The corresponding decrease in optical pulse energy at these levels suggests that most of the electrical energy is being dissipated in capacitive charging, as highlighted in Fig. 8(b), which shows the per-pixel pulse energy as a function of active pixel number.

    (a) Total chip pulse energy for 5 ns-long pulses as a function of the number of active pixels at 10 MHz, and (b) pulse energy per pixel as a function of number of active pixels.

    Figure 8.(a) Total chip pulse energy for 5 ns-long pulses as a function of the number of active pixels at 10 MHz, and (b) pulse energy per pixel as a function of number of active pixels.

    4. DISCUSSION

    The LED-on-CMOS-based DLP system demonstrates absolute frame rates in both binary and true gray scale that exceed what is possible with current DMD- or LCD-based systems, and can be modulated with pulse times down to the nanosecond range. In addition, the single chip-scale emissive display does not require the external light source and optics necessary for transmissive or reflective DLPs, making it a compact optoelectronic component, easily integrated with printed circuit board (PCB) packaging. It is worth setting its performance in context with the current state of the art, particularly in DMD-based systems, with respect to the envisaged application areas.

    The absolute frame rate of a DLP system is a crucial feature in scientific imaging applications, including spatially patterned computational imaging schemes [911,13], or where multiple functions can be realized simultaneously, for example, transmission of high-speed data on top of lower rate positioning or imaging illumination [6]. For example, a compression-free single-pixel camera with 128×128 pixel resolution requires a projection rate of at least 0.49 Mfps to provide an overall imaging rate of 30 fps. By using emissive LED pixels, our system can access switching times that are significantly faster than MEMS-based DMD pixels. A detailed comparison, in terms of pixel count and absolute frame rate, to other DLP technologies is given in Fig. 9, which includes the early stage research prototypes from Refs. [36,24,3133] and the commercial devices listed in Data File 1. The dashed diagonal lines on Fig. 9 represent effective pixel clock rate, calculated as the frame rate multiplied by the number of available pixels. It can be seen that compared to established technologies based on LCD, DMD/MEMS and previous LED arrays, the LED array presented here provides an order of magnitude improvement in frame rate at a competitive pixel count. The highest frame rate reported for a 2D light modulator, to our knowledge, is 1 Gfps, achieved with a 4×4 array at a wavelength of 1550 nm and providing phase modulation only [33]. Even though this phase modulator has been included in Fig. 9 (square data point upper left), its low pixel count and operating principles make it incompatible with the application space of the high-density intensity modulation in the visible provided by the micro-LED array. There are also 1D pixel array approaches that achieve update rates near the Mfps mark, albeit at a significantly lower pixel count [32]. We note that vertical cavity semiconductor lasers (VCSELs) can be fabricated into dense arrays, where each element offers gigahertz modulation bandwidths [34]. However, VCSEL arrays do not currently offer the pattern programmability that we present here.

    Comparison of the LED array presented here with other 2D programmable pattern sources, with pixel count and frame rate as parameters. Note the scaling potential of our technology in terms of number of pixels. Details of individual devices are provided in Data File 1.

    Figure 9.Comparison of the LED array presented here with other 2D programmable pattern sources, with pixel count and frame rate as parameters. Note the scaling potential of our technology in terms of number of pixels. Details of individual devices are provided in Data File 1.

    The absolute frame rate of our system for a binary pattern projection is 0.5 Mfps, compared with a few tens of kiloframes per second for DMD-based systems (e.g., 60 kfps for the BMC Hex 1011). For two-pattern toggling, our system rate increases to >2  MHz. For 5-bit gray-scale projection, our system can achieve 83 kfps, compared with 2  kfps for DMDs. We achieve a pixel clock rate of 8.2 Gpixels/s in continuous mode and 32.8 Gpixels/s in toggle mode, which is comparable to state-of-the-art commercial DLP systems (e.g., 61 Gpixels/s for the Texas Instruments DLP9000). Whereas DMD frame rates are limited by the mechanical switching time of the pixels, our system is currently limited by electronic interfacing, with substantial overhead in pixel switching rate still available. We note that higher data rate interfaces to the CMOS chip are possible and have been employed in high-throughput imaging chips, which have demonstrated pixel clock rates of up to 5.2 Tpixels/s [35]. Using these types of data interfaces with the LED-on-CMOS technology presented here would enable 1  Mfps at >1  Mpixels.

    In this work, we demonstrated pulsed pixel operation of 4 ns minimum duration at a repetition rate of 100 MHz, and previous LED devices have shown subnanosecond capability [22]. These time scales are compatible with time-correlated single-photon counting (TCSPC) applied, for example, to fluorescence lifetime imaging or time-of-flight ranging [8,15]. Crucially, this function is overlaid with the binary pattern projection, which allows content adaptive illumination for TCSPC and/or a combination of TCSPC with electro-optical trapping [14]. In order to combine such short pulse performance with spatiotemporal patterning, one would otherwise have to combine a lower frame-rate transmissive or reflective spatial light modulator with a pulsed laser, thus increasing complexity, footprint, and cost of the system.

    The form factor of the DLP system can also be an important factor in systems design, and the emissive nature of the device presented here presents a contrast with traditional DLP technology. The use of external light sources with modulator type DLPs allows them to employ a wide variety of sources to fit application needs, including broadband LEDs, single-frequency laser sources, or ultrashort pulsed lasers. The trade-off in this arrangement is the space requirement for external coupling optics and the sources themselves, which need to be packaged together. In contrast, the LED-based DLP can be mounted on flat-panel electronic circuit boards with only output coupling optics required. The DLP system presented here makes use of LED emitters for their high brightness, direct electronic emission control, and potential for high bandwidth modulation. The device here consumed 5 W of electrical power during the experiments shown above, which compares to a value of 10 W for a typical commercial DMD (Texas Instruments DLP9000), not including the power requirements of the external light source required.

    While the capabilities of the chip have been discussed here in the main as a source for imaging and sensing applications, it should be noted that they also present significant potential as high-speed pattern-programmable excitation sources for a wide range of experimental and engineering systems. The low size weight and power consumption metrics of the device make it particularly attractive for low-cost deployment where concurrent data communications and positioning or imaging functions are required, for example, in light-fidelity (LiFi) settings or for autonomous vehicle systems. Finally, the high-speed, highly parallel optical pattern generation could be used as a multiplexed optical pump source for physical effects including quantum-dot-based single-photon emission, nanowire photonics on-chip or spatial light modulators based on fast optical absorption modulation effects. In future, LED wavelengths from the UV to the NIR could be employed to access applications including high-speed, mask-free photolithography, or scene-adaptive IR time of flight imaging.

    5. CONCLUSION

    By employing micro-LED emitters as elements in DLP systems, a number of complementary modes of operation can be achieved, with frame update rates significantly higher than current mechanically based systems, while having lower size weight and power requirements. We have presented a smart pixel control CMOS drive chip directly bonded to an emissive micro-LED pixel chip with an active pixel count >14,000. By employing the direct current control of each pixel element, the array can be used to project patterns at rates up to 0.5 Mfps, and over 2 MHz in a two-pattern toggling mode. This represents an order of magnitude improvement over binary pattern projection using DMD displays. Furthermore, in-pixel memory enables direct gray-scale control of emission levels concurrently with binary pattern switching, without degrading frame rates. Finally, the chip can be operated with a rolling or global pixel update and in a nanosecond pulse mode with subnanosecond delay across the full array. We illustrated the potential of this system in an OCC application, demonstrating data transmission rates greater than 5 Gb/s. The operating characteristics of this system make it an attractive platform technology for future applications, including computational imaging, microscopy, and spatiotemporally controlled optical pumping of physical systems.

    Acknowledgment

    Acknowledgment. We thank Fraunhofer IZM for their help with flip-chip bonding. The authors thank Graeme Johnstone for help with the toggle mode characterization.

    APPENDIX A: DETAILED DESIGN SPECIFICATIONS AND PERFORMANCE PARAMETERS

    Table 1 gives an overview of the technical parameters of the DLP chip.

    Design Specifications and Performance Parameters

    ParameterValueAdditional Detail
    Array dimensions128×128
    Pixel pitch50 μm
    Pixel active area30μm×30μm
    Pixel fill factor36%Light-emitting area
    Frame rate0.5 MfpsLimited by 8 Gb/s digital electronic interface between external FPGA controller and LED chip
    Frame update modeGlobal/rollingBoth available irrespective of frame rate
    Pattern toggling rate2 MHz8 MHz possible at compromised duty cycle/intensity
    Rise time315 nsAll pixels off to all pixels on
    Fall time8 nsAll pixels on to all pixels off
    Frame rate (gray scale)83 kfpsLimited by 8 Gb/s digital electronic interface between external FPGA controller and LED chip
    Number of gray values325 bit
    LED current87 μASingle pixel; highest gray level setting
    Optical output power22 μWSingle pixel; highest gray level setting
    Display brightness2.8×104  cd/m2LEDs emitting at 450 nm wavelength
    Maximum pulse repetition rate100 MHz
    Minimum pulse duration4 nsLimited by external controller
    Pulse energy0.08 pJSingle pixel at 4 ns, 20 μW peak power
    17 pJPattern with 1000 active pixels at 4 ns, 1.5 A peak current

    The LED array was flip-chip bonded onto the CMOS driver chip using an indium-based bonding process, and the flip-chip bonded device was packaged and wire-bonded into a ceramic package, which can be inserted into a socket on a custom-made printed circuit board. The digital control signals to the CMOS chip are supplied by an Opal Kelly XEM7310-A200 field-programmable gate array (FPGA). A total of six LED chips were fabricated for this work, all of which emit at 450 nm. In principle, any wavelength supported by the GaN material system can be used, i.e., <280  nm  to>520  nm. The critical step in the system fabrication flow is the flip-chip bonding of the LED chip to the CMOS driver chip. The functional optical pixel yield for the six completed systems ranged between 30% and 86%, with four of the six devices having a yield above 77%, representing >12.5×103 active pixels. The pixel yield values are representative of the research laboratory fabrication processing and can be expected to be much improved using industrial foundry processing and quality control.

    The DAC was designed to provide currents ranging from 0 μA (gray level 0) to 87 μA (gray level 31). The current levels per pixel were chosen in order to reduce electrical cross talk and to keep the maximum current across the entire chip on the order of 1 A, which is a value that was found to be sustainable by previous generations of CMOS-controlled micro-LED arrays [25].

    APPENDIX B: CMOS SMART PIXEL DESIGN

    The detailed block diagram of the CMOS chip is provided in Fig. 10. The chip is implemented in 0.35 μm CMOS technology from Austria Microsystems and occupies 7.66mm×8.33  mm. Each pixel driver is connected to the anode contact of one micro-LED. All micro-LEDs have a common cathode (gnd_LED) that is separated from the ground terminal (gnd) of the driver circuit. This enables higher optical output power by pulling gnd_LED below the global ground level and biasing the micro-LEDs with higher voltage. The common cathode is connected to a wide cathode ring around the pixel grid and multiple pads. Two voltage domains are used on the chip, 3.3 V (vdd) for the control logic and 5 V (vdd_LED) inside the pixels to drive the micro-LEDs. A power grid is adopted on the pixel grid to evenly distribute the power supply and ground lines in different rows and columns.

    Schematic diagram of the signal distribution circuit of the CMOS driver chip.

    Figure 10.Schematic diagram of the signal distribution circuit of the CMOS driver chip.

    Detailed timing diagram of gray-scale and binary-pattern loading.

    Figure 11.Detailed timing diagram of gray-scale and binary-pattern loading.

    3. Pixel

    Figure 3 shows the circuit diagram of a pixel which comprises a micro-LED bump-bonded to its CMOS driver through the anode contact. The pixel driver includes 2-bit memory, 5-bit current DAC, 2 mA driver, pulse-width modulation (PWM) logic, and output stage for driving the micro-LED. The in-pixel memory stores the LED state to have global illumination of the whole array. The 2-bit memory enables writing a new datum in one memory cell (e.g., M1), while the previous datum from the other memory cell (e.g., M2) is displayed on the LED. This mode is achievable with ROL=0 while the data (PAT) can be written/displayed with both GS=0 and GS=1. In the rolling mode with ROL=1, the memory will be single bit and data will be rolled. The driver inverter drives the LED with the current generated by a 5-bit current DAC at 5 V. The bias voltage of the DAC, Vbias, is generated by the internal bias, Int_Bias, to produce 0–75 μA current range for the DAC. To produce different current ranges for the DAC, a different Vbias can be generated through the external bias, Ext_Bias, while the internal bias is disabled. In addition, a single transistor can drive the LED with higher level of current (2 mA) while applying PWM with pulse input. This allows using the array for applications that need a higher level of optical power with narrow pulse width. Although a global reset is used for initializing the chip, with Pulse=0 can ensure the output driver is OFF during the chip power up.

    Figure 4 shows a timing diagram of the data writing/displaying in a pixel. The DAC data are first generated when DAC_WR is active. Then the data pattern (PAT) is written to a memory cell while PAT_WR is active. When a new pattern is being written in M1 (or M2), the pattern data in M2 (or M1) are displayed on the LED.

    APPENDIX C: HIGH-RATE PATTERN PROJECTION EXPERIMENTS

    In order to demonstrate that a frame rate of 0.5 Mfps for binary patterns and 83 kfps for gray-scale patterns are indeed achieved, the signal “LOAD_CLK” was monitored while loading a single pattern. The signal “LOAD_CLK” is a digital signal that alternates between logic high and low for the entire duration of the pattern load process and is continuously logic low while no pattern is loaded. Figure 12(a) shows the trace of LOAD_CLK while a single binary pattern is loaded, and it can be seen that the process is completed within 2 μs. Similarly, Fig. 12(b) shows loading of a complete gray-scale pattern within 12 μs. The total amount of data transferred for a single gray-scale pattern is 5 times that of a single binary pattern. However, the loading time for a gray-scale pattern is slightly longer than 10 μs due to the overhead required for the signals “SH_DAC_CLK” and “LOAD_DAC_CLK,” which are illustrated in Fig. 11.

    Oscilloscope traces of the signal “LOAD_CLK” in Fig. 11 when (a) loading a binary pattern and (b) loading a gray-scale pattern.

    Figure 12.Oscilloscope traces of the signal “LOAD_CLK” in Fig. 11 when (a) loading a binary pattern and (b) loading a gray-scale pattern.

    Statistics from the camera communications experiment. (a) Histogram of time-average pixel intensity, (b) BER at 0.25 Mfps versus time-averaged pixel intensity, and histograms of BER at (c) 0.25 Mfps and (d) 0.4 Mfps.

    Figure 13.Statistics from the camera communications experiment. (a) Histogram of time-average pixel intensity, (b) BER at 0.25 Mfps versus time-averaged pixel intensity, and histograms of BER at (c) 0.25 Mfps and (d) 0.4 Mfps.

    Figure 13(c) shows the BER histogram at 0.25 Mfps. We see that the unbonded pixels have a BER of 0.5, and more than 76% of the pixels have a BER of 104 or less, i.e., below the sensitivity of the experiment, and more than 77% of the pixels have a BER below the 7% FEC threshold. Comparison to Fig. 11(b) verifies that the pixels with BER between 0.5 and 104 are almost exclusively pixels with a low brightness in Fig. 13(a), except for the five interconnected pixels. Figure 13(d) shows a similar BER histogram at 0.4 Mfps. Due to the marginal eye opening at this rate, the BER is more sensitive to the signal-to-noise ratio and thus the pixel intensity. Therefore, a lower percentage of 62% of the pixels achieved a BER below the sensitivity of the experiment, and more than 71% of the pixels achieved a BER below the 7% FEC threshold.

    References

    [1] A. D. Griffiths, J. Herrnsdorf, J. J. D. McKendry, M. J. Strain, M. D. Dawson. Gallium nitride micro-light-emitting diode structured light sources for multi-modal optical wireless communications systems. Philos. Trans. R. Soc. A, 378, 20190185(2020).

    [2] A. D. Griffiths, J. Herrnsdorf, M. J. Strain, M. D. Dawson. Scalable visible light communications with a micro-LED array projector and high-speed smartphone camera. Opt. Express, 27, 15585-15594(2019).

    [3] X. Li, W. K. Cho, B. Hussain, H. S. Kwok, C. P. Yue. 66-3: micro-LED display with simultaneous visible light communication function. SID Symp. Dig. Tech. Pap., 49, 876-879(2018).

    [4] X. Li, B. Hussain, J. Kang, H. S. Kwok, C. P. Yue. Smart μLED display-VLC system with a PD-based/camera-based receiver for NFC applications. IEEE Photon. J., 11, 7901008(2019).

    [5] X. Li, L. Wu, Z. Liu, B. Hussain, W. C. Chong, K. M. Lau, C. P. Yue. Design and characterization of active matrix LED microdisplays with embedded visible light communication transmitter. J. Lightwave Technol., 34, 3449-3457(2016).

    [6] J. Herrnsdorf, M. J. Strain, E. Gu, R. K. Henderson, M. D. Dawson. Positioning and space-division multiple access enabled by structured illumination with light-emitting diodes. J. Lightwave Technol., 35, 2339-2345(2017).

    [7] M. P. Edgar, G. M. Gibson, M. J. Padgett. Principles and prospects for single-pixel imaging. Nat. Photonics, 13, 13-20(2019).

    [8] A. D. Griffiths, H. Chen, D. D.-U. Li, R. K. Henderson, J. Herrnsdorf, M. D. Dawson, M. J. Strain. Multispectral time-of-flight imaging using light-emitting diodes. Opt. Express, 27, 35485-35498(2019).

    [9] B. Sun, M. P. Edgar, R. Bowman, L. E. Vittert, S. Welsh, A. Bowman, M. J. Padgett. 3D computational imaging with single-pixel detectors. Science, 340, 844-847(2013).

    [10] T. Weise, B. Leibe, L. V. Gool. Fast 3D scanning with automatic motion compensation. IEEE Conference on Computer Vision and Pattern Recognition, 1-8(2007).

    [11] Z.-H. Xu, W. Chen, J. Penuelas, M. Padgett, M.-J. Sun. 1000 fps computational ghost imaging using LED-based structured illumination. Opt. Express, 26, 2427-2434(2018).

    [12] Y. Hu, Q. Chen, S. Feng, C. Zuo. Microscopic fringe projection profilometry: a review. Opt. Lasers Eng., 135, 106192(2020).

    [13] G. M. Gibson, S. D. Johnson, M. J. Padgett. Single-pixel imaging 12 years on: a review. Opt. Express, 28, 28190-28208(2020).

    [14] A. H. Jeorrett, S. L. Neale, D. Massoubre, E. Gu, R. K. Henderson, O. Millington, K. Mathieson, M. D. Dawson. Optoelectronic tweezers system for single cell manipulation and fluorescence imaging of live immune cells. Opt. Express, 22, 1372-1380(2014).

    [15] B. R. Rae, K. R. Muir, Z. Gong, J. McKendry, J. M. Girkin, E. Gu, D. Renshaw, M. D. Dawson, R. K. Henderson. A CMOS time-resolved fluorescence lifetime analysis micro-system. Sensors, 9, 9255-9274(2009).

    [16] B. McGovern, R. B. Palmini, N. Grossman, E. M. Drakakis, V. Poher, M. A. A. Neil, P. Degenaar. A new individually addressable micro-LED array for photogenetic neural stimulation. IEEE Trans. Biomed. Circuits Syst., 4, 469-476(2010).

    [17] N. Grossman, V. Poher, M. S. Grubb, G. T. Kennedy, K. Nikolic, B. McGovern, R. B. Palmini, Z. Gong, E. M. Drakakis, M. A. A. Neil, M. D. Dawson, J. Burrone, P. Degenaar. Multi-site optical excitation using ChR2 and micro-LED array. J. Neural Eng., 7, 016004(2010).

    [18] P. F. V. Kessel, L. J. Hornbeck, R. E. Meier, M. R. Douglass. A MEMS-based projection display. Proc. IEEE, 86, 1687-1704(1998).

    [19] H. Kawamoto. The history of liquid-crystal displays. Proc. IEEE, 90, 460-500(2002).

    [20] J. Day, J. Li, D. Y. C. Lie, C. Bradford, J. Y. Lin, H. X. Jiang. III-Nitride full-scale high-resolution microdisplays. Appl. Phys. Lett., 99, 031116(2011).

    [21] M. S. Islim, R. X. Ferreira, X. He, E. Xie, S. Videv, S. Viola, S. Watson, N. Bamiedakis, R. V. Penty, I. H. White, A. E. Kelly, E. Gu, H. Haas, M. D. Dawson. Towards 10 Gb/s orthogonal frequency division multiplexing-based visible light communication using a GaN violet micro-LED. Photon. Res., 5, A35-A43(2017).

    [22] J. J. D. McKendry, B. R. Rae, Z. Gong, K. R. Muir, B. Guilhabert, D. Massoubre, E. Gu, D. Renshaw, M. D. Dawson, R. K. Henderson. Individually addressable AlInGaN micro-LED arrays with CMOS control and subnanosecond output pulses. IEEE Photonics Technol. Lett., 21, 811-813(2009).

    [23] B. R. Rae, C. Griffin, J. McKendry, J. M. Girkin, H. X. Zhang, E. Gu, D. Renshaw, E. Charbon, M. D. Dawson, R. K. Henderson. CMOS driven micro-pixel LEDs integrated with single photon avalanche diodes for time resolved fluorescence measurements. J. Phys. D, 41, 094011(2008).

    [24] J. Herrnsdorf, J. J. D. McKendry, E. Xie, M. J. Strain, I. M. Watson, E. Gu, M. D. Dawson. High speed spatial encoding enabled by CMOS-controlled micro-LED arrays. IEEE Photonics Society Summer Topical Meeting Series (SUM), 173-174(2016).

    [25] J. Herrnsdorf, J. J. D. McKendry, S. Zhang, E. Xie, R. Ferreira, D. Massoubre, A. M. Zuhdi, R. K. Henderson, I. Underwood, S. Watson, A. E. Kelly, E. Gu, M. D. Dawson. Active-matrix GaN micro light-emitting diode display with unprecedented brightness. IEEE Trans. Electron Devices, 62, 1918-1925(2015).

    [26] S. Rajbhandari, A. V. N. Jalajakumari, H. Chun, G. Faulkner, K. Cameron, R. Henderson, D. Tsonev, H. Haas, E. Xie, J. J. D. McKendry, J. Herrnsdorf, R. Ferreira, E. Gu, M. D. Dawson, D. O. Brien. A multigigabit per second integrated multiple-input multiple-output VLC demonstrator. J. Lightwave Technol., 35, 4358-4365(2017).

    [27] S. Zhang, S. Watson, J. J. D. McKendry, D. Massoubre, A. Cogman, E. Gu, R. K. Henderson, A. E. Kelly, M. D. Dawson. 1.5 Gbit/s multi-channel visible light communications using CMOS-controlled GaN-based LEDs. J. Lightwave Technol., 31, 1211-1216(2013).

    [28] W. Liu, Z. Xu. Some practical constraints and solutions for optical camera communication. Philos. Trans. R. Soc. A, 378, 20190191(2020).

    [29] I. Takai, S. Ito, K. Yasutomi, K. Kagawa, M. Andoh, S. Kawahito. LED and CMOS image sensor based optical wireless communication system for automotive applications. IEEE Photon. J., 5, 6801418(2013).

    [30] B. Razavi. The current-steering DAC [a circuit for all seasons]. IEEE Solid-State Circuits Mag., 10, 11-15(2018).

    [31] Y. Horie, A. Arbabi, E. Arbabi, S. M. Kamali, A. Faraon. High-speed, phase-dominant spatial light modulation with silicon-based active resonant antennas. ACS Photon., 5, 1711-1717(2018).

    [32] Y.-W. Huang, H. W. H. Lee, R. Sokhoyan, R. A. Pala, K. Thyagarajan, S. Han, D. P. Tsai, H. A. Atwater. Gate-tunable conducting oxide metasurfaces. Nano Lett., 16, 5319-5325(2016).

    [33] A. Smolyaninov, A. El Amili, F. Vallini, S. Pappert, Y. Fainman. Programmable plasmonic phase modulation of free-space wavefronts at gigahertz rates. Nat. Photonics, 13, 431-435(2019).

    [34] Z. Khan, J.-C. Shih, R.-L. Chao, T.-L. Tsai, H.-C. Wang, G.-W. Fan, Y.-C. Lin, J.-W. Shi. High-brightness and high-speed vertical-cavity surface-emitting laser arrays. Optica, 7, 267-275(2020).

    [35] T. Arai, J. Yonai, T. Hayashida, H. Ohtake, H. v. Kuijk, T. G. Etoh. A 252-V/lux·s, 16.7-million-frames-per-second 312-kpixel back-side-illuminated ultrahigh-speed charge-coupled device. IEEE Trans. Electron Devices, 60, 3450-3458(2013).

    [36] https://doi.org/10.15129/b0f4afe8-fa22-444e-a29e-d9b6f754930f. https://doi.org/10.15129/b0f4afe8-fa22-444e-a29e-d9b6f754930f

    Navid Bani Hassan, Fahimeh Dehkhoda, Enyuan Xie, Johannes Herrnsdorf, Michael J. Strain, Robert Henderson, Martin D. Dawson. Ultrahigh frame rate digital light projector using chip-scale LED-on-CMOS technology[J]. Photonics Research, 2022, 10(10): 2434
    Download Citation