Compact passive silicon photonic devices with high performance are always desired for future large-scale photonic integration. Inverse design provides a promising approach to realize new-generation photonic devices, while it is still very challenging to realize complex photonic devices for most inverse designs reported previously due to the limits of computational resources. Here, we present the realization of several representative advanced passive silicon photonic devices with complex optimization, including a six-channel mode (de)multiplexer, a broadband 90 deg hybrid, and a flat-top wavelength demultiplexer. These devices are designed inversely by optimizing a subwavelength grating (SWG) region and the multimode excitation and the multimode interference are manipulated. Particularly, such SWG structures are more fabrication-friendly than those random nanostructures introduced in previous inverse designs. The realized photonic devices have decent performances in a broad bandwidth with a low excess loss of <1 dB, which is much lower than that of previous inverse-designed devices. The present inverse design strategy shows great effectiveness for designing advanced photonic devices with complex requirements (which is beyond the capability of previous inverse designs) by using affordable computational resources.