Three-dimensional (3D) objects can be recorded in the hologram as interference patterns and reproduced from a hologram. Therefore, real-time electroholography promises the ultimate 3D television^[1,2]. In electroholography, a computer-generated hologram (CGH) is used as a hologram. However, the CGH needs enormous calculations. The acceleration of CGH calculation is indispensable to realize real-time electroholography.

The accelerated CGH calculations using a graphics processing unit (GPU) have been reported in some studies^[3–11]. GPU provides excellent cost performance. The software development kit facilitates the development of the program for GPU. The real-time electroholography using a GPU cluster has also been reported in some work^[12–16].

The image quality improvement methods for electroholography, such as down-sampling^[17–21], intensity accumulation^[22–24], and adaptive intensity accumulation^[25], have been reported in the literature. In our previous work, spatiotemporal division multiplexing electroholography has been proposed to prevent the remarkable deterioration in the holographic 3D video reconstructed from a 3D point-cloud model comprising many object points^[26]. A digital micromirror device (DMD) is used as a spatial light modulator (SLM) since the spatiotemporal division multiplexing electroholography requires high-speed CGH playback. The time division multiplexing electroholography^[27–29] and the viewing-zone scanning holographic display^[30] utilizing the advantage of high-speed display by DMDs have been reported. Furthermore, the real-time spatiotemporal division multiplexing electroholography using the Horn-8 system^[31] and multiple GPU cluster system^[32] has been investigated. Here, the Horn-8 system is the special-purpose computer system for electroholography. However, the system becomes large scale and very expensive.

In this Letter, we propose a spatiotemporal division multiplexing electroholographic 3D video playback from a solid-state drive (SSD). The proposed method easily realizes clear 3D video playback for a 3D point-cloud model comprising many object points.

Figure 1 shows the outline of our spatiotemporal division multiplexing method for improving the image quality of the 3D video reconstructed from the 3D point-cloud model comprising many object points. At each frame, the original 3D model is spatially divided into some sub-objects. In Fig. 1, the spatial division number of the original 3D model is denoted as N. At frame M, the respective CGHs are generated from the corresponding sub-objects, Div M-1 to Div M-N, and displayed on an SLM, on which the reconstructing light is incident. The 3D images of the respective sub-objects are reconstructed from the corresponding CGHs. The original 3D model can be observed by the persistence of vision if the 3D images of the corresponding sub-objects are reconstructed at high speed.

High-speed CGH playback can be realized using a DMD module, which consists of a DMD panel and a DMD controller. The DMD controller drives a DMD panel to display a red–green–blue (RGB) color image output from a GPU. To realize high-speed CGH playback using a DMD, “Synthesized Color CGH” is used.

Figure 2 shows how to make “Synthesized Color CGH.” At Frame M, the original 3D object is divided into six sub-objects. Here, it is assumed that the spatial division number of the original 3D model is six. The six RGB binary CGHs including “Red Binary CGH (127),” “Red Binary CGH (128),” “Green Binary CGH (127),” “Green Binary CGH (128),” “Blue Binary CGH (127),” and “Blue Binary CGH (128)” are, respectively, generated from the sub-objects, Div M-1 to Div M-6. Here, “Red Binary CGH (127)” is the binary CGH, which is drawn in two colors: black and red, with the gradation value of 127. Similarly, other RGB binary CGHs are binary CGHs drawn in two colors: black and the corresponding color with corresponding gradation values. “Synthesized Red CGH” is synthesized by “Red Binary CGH (R127)” and “Red Binary CGH (R128),” and it is drawn in four colors: black and three red colors with the gradation values of 127, 128, and 255. “Synthesized Green CGH” and “Synthesized Blue CGH” are also similarly synthesized by the corresponding binary CGHs, and these are drawn in four colors: black and the corresponding colors with the gradation values of 127, 128, and 255. Finally, “Synthesized Color CGH” is synthesized by “Synthesized Red CGH,” “Synthesized Green CGH,” and “Synthesized Blue CGH.”

In CGH calculation, we used the following equation obtained by Fresnel approximation^[3]: I(xh,yh,0)=∑j=1NobjAjcos{πλzj[(xh−xj)2+(yh−yj)2]},(1)where (xh, yh, 0) and (xj, yj, zj) are the coordinates of a point on the CGH and the object point j of a 3D object, respectively. I(xh,yh,zh) is the light intensity at a point on the CGH. Aj is the intensity of the object point j. Nobj is the total number of object points of the 3D point-cloud model. λ is the wavelength of the reconstructing light. The computational complexity of Eq. (1) becomes O(NobjNcgh), where Ncgh is the number of pixels on the CGH.

At each point on the respective RGB binary CGHs shown in Fig. 2, the light intensity, I, is calculated using Eq. (1). The binarized value is set to one, when the light intensity is more than zero. Otherwise, it is set to zero. The binary CGH is generated from the binarized value. In the respective RGB binary CGHs shown in Fig. 2, the corresponding pixel is drawn in the corresponding color when the binarized value equals one. Otherwise, it is drawn in black.

The DMD module can display RGB color images using time division multiplexing in a single-frame refresh period. The DMD module can display each color image with 8 bit depth (256 gradations) using binary pulse-width modulation (PWM) when a 24 bit color image is an output to the DMD controller^[33].

Figure 3 shows a high-speed playback by “Synthesized Color CGH.” As shown in Fig. 3, the following processes are automatically executed on the DMD module after “Synthesized Color CGH” output to the DMD module from GPU.STEP 1: “Synthesized Color CGH” is split into “Synthesized Red CGH,” “Synthesized Green CGH,” and “Synthesized Blue CGH.”STEP 2: In each of three synthesized RGB color CGHs, the corresponding synthesized color CGH is also split into the two corresponding color binary CGHs. For example, “Synthesized Red CGH” is split into “Red Binary CGH (R127)” and “Red Binary CGH (R128).”STEP 3: Six RGB binary CGHs are sequentially displayed on the DMD panel in a single-frame refresh period. Here, in Fig. 3, “Red Binary CGH (127),” “Red Binary CGH (128),” “Green Binary CGH (127),” “Green Binary CGH (128),” “Blue Binary CGH (127),” and “Blue Binary CGH (128)” are indicated by “R127,” “R128,” “G127,” “G128,” “B127,” and “B128,” respectively.

The DMD module realizes 256 gradation representations in each of RGB color using binary PWM^[33]. Figure 4 shows the binary PWM sequence patterns for three red images with gradation values of 127, 128, and 255. Three red images are provided with gradation values proportional to the percentage of time when the mirrors are in the “ON” state during the display time of the red color in a single-frame refresh period. The lighting time of the red image with the gradation value of 128 is nearly equal to the one with the gradation value of 127. Thus, the display time of each color image with the gradation value of 127 is nearly equal to the one with the gradation value of 128. Therefore, in a single-frame refresh period, six RGB binary CGHs are sequentially displayed on the DMD panel approximately at the same time interval. This technique provides high-speed playback at six times speed as fast as the refresh speed of the DMD.

In the 3D object comprising many object points, spatiotemporal division multiplexing electroholography requires high-speed CGH computation for the effect of the persistence of vision. However, the CGH calculation based on the point-cloud 3D model takes a lot of time without using a high-performance computer system. Thus, when the refresh rate of the DMD is 60 Hz, it is not easy for six binary CGHs (Fig. 2) to be calculated from the 3D object comprising many object points within 16.6 ms. We propose high-speed 3D video playback from an SSD for spatiotemporal division multiplexing electroholography. All CGHs for a holographic 3D video are calculated and stored in an SSD in advance.

Furthermore, to reduce the stored CGH data in an SSD, the packing technique of a binary CGH is used^[16]. The outline of the proposed method is shown in Fig. 5. The proposed approach requires advance preparation for the high-speed playback. In all frames of a 3D video, the following steps for advance preparation are performed per frame.Step 1: The original 3D model is spatially divided into six sub-objects.Step 2: All light intensities, I, on the corresponding CGH are calculated from each of six sub-objects using Eq. (1).Step 3: The calculated light intensities, I, are binarized by a threshold value of zero. Using a bit shift, the binarized data is packed bit-by-bit into an unsigned integer array as the packed data^[16]. The packed data is stored in an SSD.

After advance preparation, the high-speed 3D playback is executed as follows.Step 1: The packed data per frame is loaded from an SSD and is unpacked^[16].Step 2: The packed data per frame generates six RGB binary CGHs, as shown in Fig. 2.Step 3: As shown in Fig. 2, the synthesized color CGH is synthesized from six RGB binary CGHs. The synthesized color CGH is the output to the DMD controller to drive a DMD panel as an SLM.

The above process is repeated until all packed CGH data, which is stored in an SSD, is finished.

Figure 6 shows the optical setup used in the evaluation experiment for the proposed method. The laser is used as the light source, and the wavelength is 532 nm. Parallel light is obtained from the objective and collimator lenses, incident to DMD panel, and used as the reconstructed light. The incident angle θ of the reconstructed light is 24°. The computer system consisting of a personal computer (PC) and a DMD is used (Table 1). Here, these software and the coordinate data of a 3D object are installed in the same SSD. The distance between the 3D object and CGH was set to 1.0 m. We used a Canon EOS 6D as a digital camera. The experimental system is shown in Figure 7.

We investigated the still 3D image reconstructed from the 3D object “fountain,” comprising 1,064,462 points against the number of divisions. Figure 8(a) shows the original 3D object “fountain.” Figure 8(b) shows the still 3D image reconstructed from the original 3D object without using the spatiotemporal division multiplexing approach. The reconstructed 3D image has obviously deteriorated. Figures 8(c) and 8(d) show the still 3D images reconstructed from the original 3D object using three-division and six-division spatiotemporal division multiplexing methods, respectively. The greater number of divisions leads to a clearer reconstructed 3D image. From the results, we used six spatial divisions for the original 3D object “fountain.”

Figure 9 shows the display time of six 3D images reconstructed from six sub-objects in spatiotemporal division multiplexing electroholography using the high-speed playback shown in Fig. 5. Red and blue lines show the display when “VSYNC” is “on” and “off,” respectively. Here, “VSYNC ON” means that the video output from the GPU is synchronized with the refresh of the DMD panel. The display time of “VSYNC OFF” stays within a single-frame refresh period (16.6 ms). The redline shows that the display of six CGHs can be played back from an SSD within 16.6 ms.

Figure 10 shows the snapshots of the reconstructed 3D video using the proposed spatiotemporal division multiplexing electroholographic playback from an SSD. In Fig. 10, the snapshots of the reconstructed 3D video using the proposed method matched with those of the original 3D video. Table 2 shows the performance of the proposed playback when the refresh rate of the DMD is 60 Hz. Consequently, the proposed method performed playback at a speed six times faster than the refresh speed of the DMD, although the packed CGH data is stored in an SSD.

We demonstrated spatiotemporal division multiplexing electroholographic 3D video playback from an SSD. Spatiotemporal division multiplexing electroholography requires the CGH playback at six times speed as fast as the speed of the original 3D video playback when the original 3D object comprises approximately 110,000 points. The results of the performance evaluation indicate that the proposed method can play back six binary CGHs from the packed CGH data stored in a commercially available SSD within 16.6 ms. We successfully reconstructed a clear 3D video of a 3D object comprising approximately 1,100,000 points using the proposed playback for spatiotemporal division multiplexing electroholography. We considered that the proposed method significantly contributes to realizing the ultimate 3D video playback system since it is very simple and facilitates the handling of holographic video playback.