Topological insulators (TIs) are a class of materials with special topological features in energy bands. Due to the topological surface states protected by time reversal symmetry (TRS)^[1-3], three-dimensional topological insulators (3D-TIs), such as Bi₂Se₃^{[1, 4]}, Bi₂Te₃^[5] and Sb₂Te₃^[6] have attracted great attention in the past few decades. TRS can be broken by introducing magnetism, which could unlock many exotic physical phenomena, such as quantum anomalous Hall effect (QAHE)^[7], topological magnetoelectric effect^[8] and mirror magnetic monopole^[9], etc. The method of doping magnetic elements was widely adopted to introduce magnetism into TIs. Conventionally, transition elements (such as Mn, Fe, V, Co, Cr, etc.^[10–20]) were used as dopants to study the doping effect of d electrons in TIs, among which the discovery of QAHE in Cr-doped Cr_0.15(Bi_0.1Sb_0.9)_1.85Te₃ thin films was well noted^[10]. It should be stressed that the typical dopant concentration of transition elements was considerably high (from ~ 0.03 to 0.075 at.), which might complicate the structural phases and chemical homogeneity of the pristine topological materials. What is more, extremely low temperature environment, i.e., in the range of sub-kelvin, was needed for the interplay between magnetism and topological states taking effects.

Compared with transition elements, rare earth (RE) elements could be optional dopant candidates for the following reasons. On one hand, they usually possess larger local magnetic moments than transition elements, which implies the possibility of larger interactions between local magnetic moments from 4f electrons with topological states, and might result in reducing the dopant concentration to achieve the desired strength of magnetic interactions. On the other hand, the trivalent state of RE ions can substitute the trivalent Bi/Sb ions in TIs without changing the charge carrier density^[21]. Research on doping TIs with various rare earth elements (such as Ce, Sm, Eu, Gd, etc.^[22-26]) have been reported, and strong competition between anti-ferromagnetism and topological states was observed. Among all the RE elements, the isolated Ce atom has only one f electron, and it is ionized into the trivalent state (Ce³⁺) when diluted doped into other materials. This implies that Ce doped topological insulators might serve as a prototypical platform to study the interactions between f-electron and topological states and explore possible exotic physical phenomena.

In this work, we reported the electronic transport properties of Ce-doped Bi₂Te₃ thin films grown on Al₂O₃(0001) substrate by molecular beam epitaxy (MBE) with various Ce concentrations. X-ray diffraction (XRD) revealed that Ce atoms substituted the Bi sites, resulting in the chemical formula of (Ce_xBi_1–x)₂Te₃. Electronic transport measurements demonstrated Kondo effect and Fermi liquid behavior of (Ce_xBi_1–x)₂Te₃ at low temperature. Weak anti-localization (WAL) effect that is closely related to the topological properties of TIs is experimentally investigated by magnetic transport measurements and theoretically studied by using the Hikami–Larkin–Nagaoka (HLN) model^[27]. Our analysis showed that the surface conducting channels persisted to be two for x in the range from 0 to 0.02, and the conducting ability is suppressed as Ce concentration further increased, indicating the possible competition between magnetic order and topological states.

Pristine and Ce-doped Bi₂Te₃ thin films were grown on an Al₂O₃(0001) substrate by MBE, following the well-established three temperature methods^[28]. The base pressure of the MBE chamber was better than 1 × 10^–10 mbar and rose to less than 5 × 10^–10 mbar during the growth. Standard Knudsen cells were used to evaporate Ce(3N), Bi(5N) and Te(5N) sources. The flux rates of the sources were calibrated by in-situ quartz crystal microbalance (QCM). Ce doping was realized by co-evaporating Ce during the MBE growth of Bi₂Te₃ thin films, and its concentration was carefully tuned by finely adjusting the evaporating temperature of the Ce source. During the growth, Bi and Te sources were kept at 530 and 315 °C, corresponding to flux rate of 0.199 and 0.779 /s, respectively. The Ce source was kept at 1280, 1310, 1340, 1360, and 1370 °C, to grow (Ce_xBi_1–x)₂Te₃ thin films with various Ce concentration (x = 0, 0.01, 0.02, 0.03, 0.04, 0.05, respectively). ex-situ X-ray diffraction (XRD) measurements were performed to characterize the out-of-plane lattice parameters. Transport measurements were performed with the standard four-probe method by a physical property measurement system (Quantum Design PPMS-9). High purity Indium (5N) was used as electrode contacts.

Fig. 1(a) shows the crystal structure of Bi₂Te₃. Bi₂Te₃ has a layered hexagonal structure. The smallest repeating unit consists of five atomic layers with a stacking sequence of -Te(1)-Bi-Te(2)-Bi-Te(1)-, which is called a quintuple layer (QL). The interaction between neighboring QLs is the relatively weak van der Waals interaction. The thickness of all the samples was kept at 100 QLs, i.e., about 100 nm. Fig. 1(b) shows the XRD results of the (Ce_xBi_1–x)₂Te₃ samples, where only the (00L) peaks of Bi₂Te₃ are present in the spectra. The absence of other diffraction peaks in the XRD suggests that the sample surface is well oriented in the direction parallel to the (0001) plane of Bi₂Te₃. The lattice parameters c perpendicular to the sample surface for all the samples were calculated. All the c values disperse within 3.045 ± 0.007 nm for all the samples, where 3.045 nm corresponds to the lattice parameter of the pristine Bi₂Te₃ thin film. This means that the differences of c between the Ce-doped and pristine Bi₂Te₃ thin films are less than ~ 0.2 %, indicating that Ce has substituted Bi during the growth, which is consistent with the substitutional doping scenario with the RE³⁺ ion iso-electronically substituting for Bi^3+[21].

Transport measurements were performed above 3.4 K (the onset superconducting transition temperature of indium, which was used as electrode contacts) to eliminate the influence of the superconducting effects from indium. The curves of the longitudinal resistivity versus temperature ( ) for the pristine and doped thin films are shown in Fig. 2(a), where all the curves are normalized with the resistivity at 300 K and are shifted vertically for a better view. The curve for the pristine Bi₂Te₃ thin film is consistent with the results from the literature^[29], demonstrating metallic behavior from RT to the lowest temperature, which is resulted from the fact that the electronic properties of Bi₂Te₃ bulk or thin film are usually dominated by intrinsic n-type and/or p-type carriers^{[6, 30]}. The discrepancy from metallic behavior at low temperature (from 3.5 to 25 K) in occurred as cerium was doped into Bi₂Te₃, as shown in Fig. 2(b), i.e., resistivity minimum developed with Ce doping. Due to the large local magnetic moments of diluted Ce³⁺ ions in Bi₂Te₃, Kondo effect^[31] would be a straightforward explanation of the occurrence of resistivity minimum at a low temperature, which contributed an increasing logarithmic term in resistivity as temperature decreased. In order to simulate the resistivity behaviors at low temperature, the following expression was used to fit the normalized (with the resistivity at 25 K being set to 1) experimental curves between 3.5 and 25 K:

${\rho _{xx}} = {\rho _{\rm{0}}}{\rm{ + }}A {T^2} - B {\rm{log}} T,$ (1)

where is the residual resistivity, and represent contributions from Fermi liquid behavior and Kondo effect at low temperature, respectively. For pristine Bi₂Te₃, considering there would be no Kondo effect, the last term was set to be zero. The fitted parameters , A and B are listed in Table 1, and the corresponding relationships are demonstrated as solid curves in Fig. 2(b).

As shown in Table 1, as the dopant concentration increases, the Fermi liquid behavior and the Kondo effect compete with each other, indicated by the gradual decrease of A and increase of B. In general, the low temperature transport behavior of (Ce_xBi_1–x)₂Te₃ films is mainly determined by the Kondo effect of Ce and the Fermi liquid behavior of electrons. The more the dopant, the stronger the Kondo effect and the weaker the Fermi liquid behavior.

Figs. 2(c) and 2(d) show the evolution of normalized magnetoresistance (MR) with Ce concentration x and temperature for x = 0.04, respectively, with MR being defined as:

$\text{MR} = \frac{{R\left( B \right) - R\left( 0 \right)}}{{R\left( 0 \right)}} \times 100\% ,$ (2)

where R(B) is the resistance measured under magnetic field B, R(0) is the resistance measured under zero field. Similar to the previous report^[32], all samples have very low MR. WAL is a destructive quantum interference effect caused by two electron scattering trajectories protected by time reversal symmetry, which stems from the π berry phase^[33]. Fig. 2(c) shows that the magnetoresistance near zero field increases rapidly with increasing field, and then gradually increases linearly under high field. This trend of magnetoresistance is a distinctive feature of weak anti-localization (WAL)^[33], and WAL is gradually suppressed as the dopant concentration increases. It implies the possibilities that the introduced Ce impurity destroys the time reversal invariant symmetry which is crucial to the topological properties of the pristine Bi₂Te₃ thin film.

The effect of WAL in MR can be described by the HLN model^[27]. Figs. 3(a) and 3(b) show the magnetic conductance of samples with different concentration. For an ideal TI surface, the change of conductance under field B can be expressed as:

$\Delta {G_{xx}}\left( B \right) \equiv G\left( B \right) - G\left( 0 \right) \cong \alpha \frac{{{e^2}}}{{2{\pi ^2}\hbar }}\left[ {\psi \left( {\frac{1}{2} + \frac{{{B_{\phi} }}}{B}} \right) - \ln \left( {\frac{{{B_{\phi} }}}{B}} \right)} \right],$ (3)

where is magnetoconductance, e is the electronic charge, is the reduced Planck constant, is the digamma function and is a characteristic magnetic field expressed as , where is the phase coherence length. As demonstrated in Figs. 3(a) and 3(b), all experimental conductance data can be fitted well by the HLN formula. The coefficient α and are extracted by fitting the HLN model, and the results are listed in Table 2 and also shown in Figs. 3(c) and 3(d). The fitting results show that gradually decreases with increasing concentration and temperature. This effect is expected because the dopant introduces disorder and magnetism to the system. The value of α is between –0.6 and –1, similar to the results of Yu et al.^[34]. And the α can be further expressed by:

$\alpha = N a,$ (4)

where should be −0.5 for the topological surface states of symplectic universality class, and N is the number of independent conducting channels^[27]. The relatively large values of N and α may be caused by the thick film's 3D WAL effect and multiple effective surfaces^[32]. The fitting results show that the number of conducting channels is about 2 for x = 0, and it still maintains the same state for x = 0.01. However, the surface conducting channels are reduced for x > 0.01. It may indicate that the transportation ability of topological surface states has been suppressed for x > 0.01, which suggests there is a strong competition between magnetic order and the topological surface states for x = 0.01–0.02. The change of N and α in the doped samples suggested that Ce may have brought certain magnetic order into the TI film, partially suppressing the topological surface states while not creating any additional bulk conducting channel in the body state. The relation between and temperature can be expressed by a power law^[35]:

${l_{\phi} } = A {T^m},$ (5)

where T is the temperature and the factor m is related to the type of dominant mechanism of WAL. It is reported that for two-dimensional e–e scattering, the power law dependence should be ; while for three-dimensional scattering, it is ^{[36, 37]}. Here, the value of m is between –0.3 and –0.6. This further confirms that the WAL effect is still dominated by the two-dimensional Nyquist e–e interaction though the TSSs is partially suppressed by the dopants Ce.

(Ce_xBi_1–x)₂Te₃ (x = 0, 0.01, 0.02, 0.03, 0.04, 0.05) monocrystalline films with a thickness of 100 nm were successfully grown on sapphire substrates by MBE, and the electrical transport properties have been investigated. The resistivity shows metallic behavior at T > 25 K and semiconductor behavior when between 5–25 K. This resistivity behavior at low temperature is mainly due to the Kondo effect of Ce and the Fermi liquid behavior of electrons, which are confirmed by Kondo theory fitting. The contribution of Kondo effect increases with increasing Ce concentration, while the contribution of Fermi liquid decreases. Besides, WAL effect was also observed from the magnetoresistance and magnetoconductance. From the fitting of the HLN model, we confirm that the WAL effect is mainly dominated by the two-dimensional Nyquist e-e interaction. The phase coherence length and the number of conducting channels extracted from HLN fitting decrease as the concentration increases, which indicates a higher concentration of dopant may further suppress or even damage the topological properties of the TI. This indicates that there is a strong competition between the topological surface states and the magnetic order introduced by dopant Ce. Through doping Ce, this work successfully introduces RE elements into topological materials and preliminary investigates its influence on the transportation properties. It may lay the foundation for further research on the influences of RE elements on topological materials, which may open the door to the study of exotic physical properties, such as QAHE of TIs at elevated temperatures.

This work was supported by the Key Research and Development Program of China (No. 2017YFA0303104), the SPC-Lab Research Fund (No. WDZC201901) and the National Science Foundation of China (No. U1630248).