Multimode optical systems based on a wide variety of optical fibers that support modes or mode groups (MGs) with desirable characteristics have become increasingly important research topics in both classical and quantum photonic information systems. Mode-division multiplexing (MDM), which utilizes multiple optical modes in one guiding fiber core as independent data communication channels to provide high capacity density, has been widely considered one of the promising solutions to overcome the capacity crunch of conventional single-mode fiber (SMF)-based communication systems and networks [1,2]. However, attempts to upscale MDM systems by introducing more mode channels are often limited by the fact that the inter-mode coupling (i.e., crosstalk) necessitates multiple-input multiple-output (MIMO) digital signal processing (DSP), and the DSP complexity increases dramatically as more modes and larger differential mode delay (DMD) are involved [3], for instance, full $6\times 6$ MIMO and $20\times 20$ MIMO DSP are required in 3-mode and 10-mode multiplexed few-mode fiber (FMF) transmission systems to compensate for the mode coupling, respectively [4,5]. In order to reduce the DSP complexity, weakly coupled FMF-based MDM schemes have been proposed, in which low coupling can be maintained between adjacent modes or MGs with relatively large differential effective refractive index ($\mathrm{\Delta }{n}_{\mathrm{eff}}$) after fiber transmission over a certain distance, enabling partial-MIMO or MIMO-free schemes that increase multiplexed mode channels with low MIMO complexity [6–9]. Nevertheless, these complexity reduction schemes are not scalable for the high-order MGs, as the number of near-degenerate modes in each MG of the graded-index FMFs increases with the MG order due to the emergence of higher-radial-order modes, resulting in higher DSP complexity for the higher-order MGs. In step-index FMFs, on the other hand, small $\mathrm{\Delta }{n}_{\mathrm{eff}}$ values can exit between high-order radial modes, resulting in relatively high crosstalk and limiting transmission distance.

Recently, ring-core-fiber (RCF)-based MDM systems that can stably support linearly-polarized (LP) or orbital angular momentum (OAM) mode sets have been reported to demonstrate promising potentials in improving the capacity of MDM systems with low DSP complexity [10–26]. Different from the conventional FMFs, the number of modes including dual polarizations in each high-order MG (MG $\mathrm{order}>0$) of the single-radial-mode RCF is fixed at 4, and the inter-MG coupling can be decreased significantly as RCFs $\mathrm{\Delta }{n}_{\mathrm{eff}}$ between the MGs increases with their azimuthal mode order [10–13]. RCF-based optical amplifiers can provide low differential modal gain for multimode amplification due to the very similar profiles between different modes [14,15]. These characteristics make RCF-based MDM systems potentially more scalable towards the higher-order mode space, which is highly attractive in future high-capacity optical fiber transmission systems.

Since the first experimental demonstration of MDM transmission over specially designed OAM fiber [16], extensive efforts have been devoted to achieving higher capacity and longer transmission distances [11,12,17–26]. Many MDM transmission demonstrations over RCFs with fiber lengths of 0.1–1 [17–21], 10 [12], 18 [11,22], 24 [25], 50 [23,26], and 100 km [24] have been reported recently, which are summarized in Table 1. These reported experiments can be divided into the MIMO-free and MIMO-based categories.

The MIMO-free transmission experiments over RCFs include nondegenerate mode multiplexing [17,18] and MG multiplexing (MGM) [19–24]. Both rely on large $\mathrm{\Delta }{n}_{\mathrm{eff}}$ to suppress the crosstalk between MGs or nondegenerate modes. Using a polarization-maintaining elliptical RCF with $\mathrm{\Delta }{n}_{\mathrm{eff}}>1\times {10}^{-4}$ among LP vector modes, MIMO-free 32-GBaud quadrature phase shift keying (QPSK) transmission was demonstrated over 0.9-km polarization-maintaining elliptical RCF using 6 LP vector modes as independent data channels [17,18]. The transmission distance of such a scheme is likely to be limited due to the limited $\mathrm{\Delta }{n}_{\mathrm{eff}}$ achievable that results in relatively high crosstalk between the mode channels. To support MGM, some RCFs have been designed to minimize the $\mathrm{\Delta }{n}_{\mathrm{eff}}$ between the intra-MG modes to around $1\times {10}^{-5}$ in order to suppress frequency-selective fading. MIMO-free MGM transmission experiments have been reported based on 2-LP MGs, each carrying 10-Gb/s on–off keying (OOK) signal over 0.1-km graded index (GI-) RCF [19] and 1-km step index (SI-) RCF [20], respectively. The scheme was also extended to 3-MG multiplexed transmission over a 0.36-km GI-RCF [21]. In our research group, an MIMO-free OAM-MGM transmission scheme using a maximal-ratio combining algorithm-based receive-diversity receiver was also reported [11], in which 3-MG multiplexed 100-Gb/s and 2-MG multiplexed 40-Gb/s quadrature amplitude modulation (QAM) discrete multi-tone (DMT) signals are respectively transmitted over 1-km and 18.4-km GI-RCFs. MGM has also been combined with wavelength-division multiplexing (WDM) transmission with coherent detection to achieve 8.4-Tbit/s capacity over an 18-km GI-RCF, in which two OAM modes ($l=+4$ and $+5$) of two MGs with 112 WDM channels are multiplexed to give 224 data channels [22]. Recently, MIMO-free 15-Gbit/s and 128-Gbit/s OOK MGM transmissions over 50-km [23] and 100-km [24] refractive-index-profile (RIP)-modulated RCFs, respectively, have also been demonstrated. Due to the crosstalk or the limited number of available channels, MIMO-free transmission schemes over RCFs are more suitable for low-cost short-reach MDM systems.

Compared with MIMO-free schemes, MIMO-based MDM schemes can significantly increase the capacity-distance product in RCF-based systems. MDM transmissions at raw data rates of 1.12 (Tbit/s)/$\lambda$ over 1 km and 560 (Gb/s)/$\lambda$ over 24 km RCFs have been demonstrated [25], which required high-complexity $10\times 10$ MIMO processing due to the high modal crosstalk in the mode (de)multiplexer as well as relatively high coupling among multiplexed MGs resulting from their small $\mathrm{\Delta }{n}_{\mathrm{eff}}$. Exploiting the very low inter-MG coupling and degenerate intra-MG modal characteristics of purposed-designed RCFs, our research group previously proposed a scalable modular $4\times 4$ MIMO scheme in which only modular $4\times 4$ MIMO equalization with relatively low complexity is required to suppress the intra-MG crosstalk in RCF-based MDM systems [12]. We first implemented such an MDM transmission experiment over a 10-km GI-RCF, utilizing MGs $|l|=4$ and $|l|=5$ that each contain four OAM modes and that are decoupled by the large inter-MG $\mathrm{\Delta }{n}_{\mathrm{eff}}$ [12] to demonstrate 5.12-Tbit/s OAM-MDM/WDM transmission with 32-GBaud QPSK signals. More recently, we also demonstrated 2.56-Tbit/s OAM-MDM/WDM [26] over a 50-km RCF with a modulated refractive index profile (RIP) [23]. While making steady progress, the single-span transmission distances of these RCF-based MDM experiments were delimited by the relatively high attenuation and inter-MG coupling efficiency found in RCFs compared with those of conventional FMFs.

In this work, by employing a specially designed and fabricated RCF with low attenuation and very low inter-MG crosstalk [24], we report experimental demonstration of OAM-MDM/WDM communications at the 100-km milestone over a single-span RCF without optical amplification. The transmission of eight OAM mode channels belonging to two adjacent OAM MGs $|l|=2$ and $|l|=3$, each mode channel carrying 10 WDM channels with 16-GBaud QPSK modulation, achieved an aggregate capacity of 2.56 Tbit/s and an overall spectral efficiency of 10.24 bit/(s · Hz) (defined as capacity/WDM bandwidth) with the bit-error rate (BER) below the 20% soft-decision forward error correction (FEC) threshold of $2.4\times {10}^{-2}$. A record capacity-distance product of 256 (Tbit/s) · km is successfully achieved for RCF-based OAM-MDM systems. The system only uses small-scale modular $4\times 4$ MIMO processing to recover the signals in each MG after the 100-km RCF transmission, without MIMO processing between the channels belonging to different MGs.

The novel RCF was specifically designed and fabricated to enable low-loss and low-inter-MG crosstalk transmission of high-order OAM MDM channels [24]. The cross section and RIP of the RCF are depicted in Figs. 1(a) and 1(b), respectively. The RIP of the RCF is modulated with an RI notch at the top of the ring core, which suppresses the micro-perturbation-induced inter-MG coupling [24,27]. Table 2 shows the characteristics of the fabricated RCF, which supports four azimuthal MGs ($|l|=0$ to $|l|=3$). It can be seen that $\mathrm{\Delta }{n}_{\mathrm{eff}}>0.7\times {10}^{-3}$ has been achieved between all adjacent MGs, increasing as the MG order increases. The two adjacent high-order MGs $|l|=2$ and $|l|=3$ with $\mathrm{\Delta }{n}_{\mathrm{eff}}=2.5\times {10}^{-3}$ are chosen in the transmission experiment. The crosstalk between these two MGs is $<-36\mathrm{dB}/\mathrm{km}$ [24,27]. The attenuation of the RCF, measured by an optical time-domain reflectometer (OTDR) and shown in Fig. 1(c), is around 0.21 dB/km with mode-dependent loss (MDL) of $<0.03\mathrm{dB}/\mathrm{km}$ among all MGs. The measured differential group delay (DGD) relative to the MG $|l|=0$ is also listed in Table 2.

The experimental setup for the OAM-MDM/WDM data transmission is illustrated in Fig. 2. Ten optical carriers from narrow-linewidth tunable lasers are combined by a wavelength-division multiplexer, with wavelengths ranging from 1549.32 to 1551.12 nm in a 0.2 nm/25-GHz grid. Subsequently, the 10 WDM carriers are modulated by 16-GBaud QPSK signals from an arbitrary waveform generator (AWG) through an in-phase/quadrature (I/Q) modulator. It is noted that the adjacent WDM carriers are modulated with the same electrical signals due to the hardware limitation in our lab. Little impact on the system performance has been observed when sufficiently large guard bands exist between adjacent WDM channels, since the inter-channel linear crosstalk rather than nonlinearity dominates the signal-to-noise ratio (SNR) performance of the received signals in the system. The sample rate of the AWG is 64 GSa/s, and the modulated electrical data sequence is pseudo-random binary sequence (PRBS) with pattern length of $2^{18}$.

The WDM signals are split into four branches, which are separately delayed for data pattern decorrelation. After optical amplification by erbium-doped fiber amplifiers (EDFAs), collimation, and linearly polarization, they are grouped into two pairs, each pair reflected by a phase-only spatial light modulator (SLM) for the conversion to OAM modes of $l=+2$, $+3$ or $l=-2$, $-3$. Each pair of OAM modes, with one of them passing a half-wave plate (HWP) for a 90-deg rotation of polarization, is combined by a polarization beam combiner (PBC) as orthogonally polarized channels to generate the four OAM modes of $⟨+2,X⟩$, $⟨-2,X⟩$, $⟨+3,Y⟩$, and $⟨-3,Y⟩$. Here $X$ and $Y$ represent the horizontal and vertical polarization, respectively. Then the four OAM modes of two pairs are combined together by a beam splitter (BS). After that, the combined OAM modes are converted into circular polarizations by a quarter-wave plate (QWP) and polarization multiplexed by using a polarizing beam splitter (PBS) with an optical delay path and a PBC, so the four modes of each MG of $l=\pm 2$ or $l=\pm 3$ with dual polarizations are generated and multiplexed. The eight multiplexed OAM modes are converted into circular polarizations and coupled into the 100-km RCF.

In Fig. 2, insets (i) and (ii) show the observed intensity profiles of MGs $|l|=2$ and $|l|=3$ after 100-km RCF transmission, respectively. After fiber transmission, all output modes from the fiber are collimated and converted back to horizontal or vertical linear polarizations before being split into two branches. Each of the two branches passes through a commercial vortex phase plate (VPP) with opposite topological charge signs ($+/-l$). Here only one OAM MG is demultiplexed and converted to Gaussian beams by VPPs at a time. For each detected MG ($|l|=2$ or 3), the two Gaussian beams emanating from the VPPs, corresponding to $+l$ and $-l$ OAM modes each containing two polarization multiplexed signals, are simultaneously coupled into two SMF-pigtailed dual-polarization integrated coherent optical receivers (ICRs). The eight output electrical waveforms (including the I/Q for each mode) from the ICRs are simultaneously recorded by an eight-channel real-time oscilloscope (OSC, Teledyne LeCroy 10-36ZI) operated at 80 GSa/s. The offline digital signal processing (DSP) includes timing phase recovery, $4\times 4$ MIMO equalization based on the conventional blind constant modulus algorithm (CMA), frequency offset compensation, and carrier phase estimation, after which the BERs are finally evaluated. The measurement is repeated for each multiplexed MG as well as each multiplexed wavelength. Although all of the supported MGs of the RCF can be used for data transmission, the relatively high crosstalk between the MGs $|l|=0$ and $|l|=1$ (resulting from their relatively small $\mathrm{\Delta }{n}_{\mathrm{eff}}$) would necessitate $6\times 6$ MIMO equalization to decouple all six vector modes in MGs $|l|=0$ and $|l|=1$ after 100-km RCF transmission, which is beyond the hardware limitation in our lab. RCFs with higher numerical apertures have been reported to support more OAM MGs [28]. As they continue to improve in simultaneously achieving a large number of MGs, low attenuation, and low inter-MG crosstalk, the modular MIMO scheme can be further extended to utilize the increased mode channels with low cost and low complexity.

The crosstalk between the two used MGs in the subsystem consisting of the 100-km RCF and the OAM (de)multiplexer (Mux/Demux) devices (i.e., the blue dashed box in Fig. 2) is first characterized based on the power measurement. As shown in Table 3, inter-MG crosstalk of around $-12\mathrm{dB}$ is obtained between MGs $|l|=2$ and $|l|=3$ over the 100-km RCF transmission system. Note that the measured inter-MG crosstalk shown in Table 3 is higher than the inter-MG coupling of the RCF ($<-36\mathrm{dB}/\mathrm{km}$ or $-16\mathrm{dB}/100\mathrm{km}$, measured by a time-domain impulse response method [27]), which is dominated by the crosstalk at the spatial (de)multiplexers.

After the 100-km RCF transmission, to recover the signals from the four strongly coupled mode channels within each MG coupling, modular $4\times 4$ MIMO equalization is performed for each transmitted MG. No MIMO equalization is applied between the MGs. Figures 3(a) and 3(b) show the convergent absolute values of tap weights of the 16 finite impulse response (FIR) filters in the two $4\times 4$ MIMO equalizer modules after 120 iterations of updating using the CMA algorithm for MGs $|l|=2$ and $|l|=3$ over 100-km RCF, respectively. One can see that the number of taps in each FIR filter is set to 25, which is sufficient to cover the small intra-MG DMD. Note that the DSP complexity could be further reduced by using the frequency-domain MIMO equalization algorithm.

To evaluate the transmission performance of 100-km RCF OAM-MDM system, BER measurements are carried out for both single-wavelength and WDM transmission scenarios. The BERs as a function of optical signal-to-noise ratio (OSNR) for all multiplexed OAM modes of MGs $|l|=2$ and $|l|=3$ at 1550.12 nm are plotted in Figs. 4(a) and 4(b), respectively. For concision, the BER values of the two orthogonal polarization OAM modes with the same $l$ are averaged and shown together. As a result, two BER curves are presented for each MG ($l=\pm 2$ or $l=\pm 3$) measurement in the cases with or without inter-MG crosstalk. The BERs of all OAM modes of MGs $|l|=2$ and $|l|=3$ are below the 20% soft-decision FEC threshold of $2.4\times {10}^{-2}$ when the OSNRs are no less than 12 dB and 14 dB, respectively. The post-transmission constellation diagrams of the recovered 16-GBaud QPSK signals of all simultaneously transmitted OAM mode channels (with inter-MG crosstalk) of MGs $|l|=2$ and $|l|=3$ at an OSNR of 24 dB are also shown in Figs. 4(c) and 4(d), respectively.

Figures 5(a) illustrates the measured BER values of the 10 WDM channels for both transmitted MGs after simultaneous transmission in the 100-km RCF OAM-MDM/WDM system. The corresponding optical spectra of 10 WDM channels at the transmitter side are presented in Fig. 5(b). It can be seen that the BERs of MGs $|l|=2$ and $|l|=3$ with a total of 80 channels after 100-km RCF transmission are below the 20% soft-decision FEC threshold of $2.4\times {10}^{-2}$, achieving an aggregate capacity of 2.56 Tbit/s, an overall spectral efficiency of 10.24 bit/(s · Hz), and a capacity-distance product of 256 (Tbit/s) · km.

In this paper, using a novel RCF with low attenuation and low inter-MG crosstalk, a two-dimensional wavelength- and mode-division-multiplexed transmission experiment involving ten wavelengths and eight OAM modes belonging to two adjacent MGs $|l|=2$ and $|l|=3$ has been implemented over a distance of 100 km, transmitting 16-GBaud QPSK signals over all 80 channels. An aggregate capacity of 2.56 Tbit/s, an overall spectral efficiency of 10.24 bit/(s · Hz), and a capacity-distance product of 256 (Tbit/s) · km have been realized. To the best of our knowledge, this is the first uninterrupted single-span OAM fiber communication experiment at 100-km distance and thus achieves the highest capacity-distance product so far for OAM fiber communications. The system employs a modular $4\times 4$ MIMO processing scheme that is scalable to higher channel counts without sacrificing signal processing simplicity, which is enabled by the low inter-MG crosstalk of the RCF.