While deep learning has demonstrated tremendous potential for photonic device design, it often demands a large amount of labeled data to train these deep neural network models. Preparing these data requires high-resolution numerical simulations or experimental measurements and cost significant, if not prohibitive, time and resources. In this work, we present a highly efficient inverse design method that combines deep neural networks with a genetic algorithm to optimize the geometry of photonic devices in the polar coordinate system. The method requires significantly less training data compared with previous inverse design methods. We implement this method to design several ultra-compact silicon photonics devices with challenging properties including power splitters with uncommon splitting ratios, a TE mode converter, and a broadband power splitter. These devices are free of the features beyond the capability of photolithography and generally in compliance with silicon photonics fabrication design rules.