The ultracold molecule is a promising candidate for versatile quantum tasks due to its long-range interaction and rich internal rovibrational states. With the help of the cavity quantum electrodynamics (QED) effects, an optical cavity can be employed to increase the efficiency of the formation of the photoassociated molecules and offers a non-demolition detection of the internal states of molecules. Here, we demonstrate the production of the high-finesse optical fiber microcavity for the Rb2 molecule cavity QED experiment, which includes the fabrication of fiber-based cavity mirrors, testing, and the assembly of ultra-high vacuum-compatible optical fiber microcavity. The optical fiber microcavity offers high cooperativity between cavity mode and ultracold molecule and paves the way for the study of molecule cavity QED experimental research.