The rapid development of cavity optomechanical systems has attracted extensive attention, and these systems are widely used in quantum information and precision measurement. In recent years, three-cavity optomechanical systems have attracted considerable attention. Compared with the single-mode optomechanical system, the multiple-mode system provides significantly higher flexible controllability. Furthermore, optomechanically induced transparency and four-wave mixing have been research hotspots in different optomechanical systems. For example, the four-wave mixing in a hybrid optomechanical system is theoretically investigated, which has important implications for the nonlinear optical properties. It is of great significance for theoretically exploring the transmission spectrum and four-wave mixing in a three-cavity optomechanical system.

The hybrid optomechanical system consists of a microwave cavity a with resonance frequency ωa and an optical cavity c₁ with resonance frequency ωc1, which are coupled to a common mechanical resonator b, while an optical cavity c₂ with resonance frequency ωc2 is coupled to the optical cavity c₁. A strong pump laser beam E_e with frequency Ωe is applied to the microwave cavity a. A weak probe laser beam E_p with frequency Ωp and a strong pump laser beam E_o with frequency Ωo are applied to the optical cavity c₁ simultaneously. In the rotating frame of the pump fields with frequency Ωe and Ωo, the whole Hamiltonian of the system is obtained. According to the Heisenberg equation and making the ansatz, we finally obtain the transmission spectrum and the four-wave mixing spectrum intensity. Then, we investigate how the evolutions of the transmission spectrum and the four-wave mixing spectrum are affected by the coupling strength and the frequency of the mechanical resonator.

When the optical cavity c₂ is absent in the hybrid optomechanical system (J=0), the transmission spectrum of the probe field shows a Lorentzian line shape. However, when J≠0, the Lorentzian peak splits into two symmetrical peaks and a transparent window occurs (Fig.2). It is clear that with the increase in the coupling strength from J=0.5κ_o to J=2κ_o, the distance between the peaks increases, and the peak value of the transmission spectrum of the probe field also increases (Fig.3). We depict the variation of the four-wave mixing spectrum with the detuning of the probe field-cavity field Δp for J=0, 0.5κ_o, κ_o, 2κ_o when ω_m=2π×5.6 MHz. In the case of Δc1=Δc2=Δa=0, when J=0, the four-wave mixing spectrum has three peaks. A Lorentzian peak locates at Δp=0 and two splitting peaks locate on both sides of the Lorentzian peak [Fig.4(a)]. With the choice of J≠0, a significant change in the four-wave mixing spectrum can be observed. The Lorentzian peak splits into two peaks. As the coupling strength J increases, the peak value decreases greatly and the distance between the peaks gradually increases [Figs.4(b)-(d)]. Next, we study how the evolution of the four-wave mixing spectrum is affected by the frequency of the mechanical resonator while the optical cavity c₂ is not in the hybrid optomechanical system (J=0). It can be seen that as the frequency of the mechanical resonator increases, the peak value of the four-wave mixing spectrum gradually decreases. Further, the peaks on both sides of the four-wave mixing spectrum located at ±Δp just correspond to the frequency ω_m of the mechanical resonator (Fig.5). Moreover, we investigate the four wave mixing spectrum as a function of the detuning Δp for the frequencies of mechanical resonator of ω_m=2π×4.6 MHz, ω_m=2π×5.6 MHz, and ω_m=2π×6.6 MHz when J=κ_o. As the frequency of the mechanical resonator increases, the peak value and the distance between the two symmetrical splitting peaks in the middle decrease. The positions of the peaks on both sides exactly correspond to the three different frequencies of the mechanical resonator. The results show that the four-wave mixing can be tuned by the frequency of the mechanical resonator (Fig.6).

We investigate the optomechanically induced transparency and four-wave mixing in a hybrid optomechanical system composed of two optical cavities, a microwave cavity, and a mechanical resonator. When the microwave cavity is driven to the red sideband and the two optical cavities are driven to the blue sideband, the optomechanically induced transparency can be changed by changing the coupling strength between the two optical cavities. Furthermore, the four-wave mixing spectrum can be modulated by controlling the coupling strength between the two optical cavities and by changing the frequency of the mechanical resonator during resonant detuning Δp=0. At the same time, a nonlinear optical method for measuring the frequency of a mechanical resonator is provided. The positions of the peaks on both sides of the four-wave mixing spectrum correspond exactly to the frequency of the mechanical resonator. These results have important significance and application prospects in quantum sensing and quantum information processing.