Significance Light field imaging expands classical optical imaging and provides possibilities for advancement in imaging technology. It has continued to become a major research interest in the field of computational imaging. While, objects and scenes in nature are all three-dimensional (3D) entities, and traditional imaging systems only record two-dimensional (2D) images. From geometry, traditional imaging is equivalent to the 2D projection on the image plane of a 3D object in space. Therefore, the depth information is lost during projection. To restore the object, or perform quantitative analysis on the shape, position, and internal structure of the object, we reconstruct the missing depth information and 3D structure from the 2D images. This process and related techniques are referred to as 3D imaging and have become an essential support technique with applications in areas such as biological imaging, industrial inspection, automatic navigation, and virtual reality. Among several methods for 3D imaging, light field imaging is a major approach.

Progress This paper introduces the basic theory of light field while reviewing common systems for light field capture. Key techniques and typical works in light field 3D imaging according to the categorization are discussed. For emphasis, this paper limits its discussion to geometric optics, thus only paying attention to the intensity distribution of rays in 3D space, i.e., the geometric light field. A light field refers to the distribution of radiance carried by rays in 3D space. For monochromatic and static cases, a light field is described using a five-dimensional (5D) plenoptic function. Since radiance remains unchanged along a ray unless blocked, the 5D function reduces to four-dimensional (4D) function in free space. The major challenge choosing a representation for the 4D light field is parameterizing the space of oriented rays. The most common model parameterizes rays using their intersections with two parallel planes ( Fig. 1). The advantages of this representation are that planes can be placed at infinity, and then rays are parameterized by a position and a direction, which is called the parameter/state space of rays. A point in the state space corresponds to a ray in the light field; therefore, phase space (also referred to as ray space) is used to represent the light field ( Fig. 3).

From the definition of a light field, we need to record different ray directions passing through any point in 3D space. During imaging using an ordinary camera, only the ray of one direction for each point can be recorded. Therefore, an ordinary camera used in the light field capturing should expand in dimensions such as time, space, and aperture to record rays from multiple directions. Three systems emerge from this (Table 1). They include sequential acquisition (Fig. 6), camera array (Fig. 7), and light field camera (Figs. 8--10). For research in light field imaging, different systems are flexibly selected according to a specific application.

The light field carries 3D information of the object and scene. Thus, 3D imaging is realized by modeling and processing the light field data. 3D light field imaging techniques are summarized into two categories: the light field depth estimation and the light field 3D reconstruction. The light field depth estimation obtains the depth (near or far) information about the object. A typical process of light field depth estimation starts with estimating the initial depth map with the appropriate algorithm, and then employing a global optimization or local smoothing algorithm to refine the depth map. The initial depth estimation for a light field is divided into two categories according to the different mechanisms: the method using multi-view stereo (MVS) (Figs. 11--12) and the method using the epipolar plane image (EPI) (Fig. 13). Generally, research on light field depth estimation tends to solve some open problems, e.g., modeling, processing of occlusion, depth estimation of discontinuous surfaces, depth estimation of non-Lambert surfaces, selecting algorithms for depth estimation according to the application, and improving the time efficiency of the algorithm.

When applied to measurements, such as 3D positioning and 3D point cloud generation, light field 3D reconstruction is used to obtain the true 3D coordinates. Note that light field 3D reconstruction follows the same theoretical basis as classical binocular and multiview 3D reconstruction, which is hinged on the principle of triangulation. The 3D coordinates are calculated from the intersection of rays in different directions in the 3D space. The light field 3D reconstruction can be divided into active (Fig. 14) and passive (Fig. 15) approaches according to whether the structured illumination is exploited.

Conclusions and Prospect Benefiting from the development of photonic and micronano techniques, a series of progress in the research of light field imaging systems and mechanisms has emerged recently. Due to the adoption of new techniques and devices, the quality and structure of the light field data obtained by the new systems are inevitably different from those of traditional systems, which bring new challenges to light field information processing. Depth estimation and 3D reconstruction using a novel light field imaging system are problems worthy of attention. The light field 3D imaging is the support technique for light field imaging. With the scope extension and complex increase in applications, the importance of light field 3D imaging has become increasingly prominent.