Indium phosphide (InP) nanowires (NWs) have attracted significant attention due to their exotic properties that are different from the bulk counterparts, and have been widely used for light generation, amplification, detection, modulation, and switching, etc. Here, high-quality InP NWs were directly grown on a quartz substrate by the Au-nanoparticle assisted vapor-liquid-solid method. We thoroughly studied their nonlinear optical absorption properties at 1.06 μm by the open-aperture Z-scan method. Interestingly, a transition phenomenon from saturable absorption (SA) to reverse saturable absorption (RSA) was observed with the increase of the incident laser intensity. In the analysis, we found that the effective nonlinear absorption coefficient (βeff～?102cm/MW) under the SA process was 3 orders of magnitude larger than that during the RSA processes. Furthermore, the SA properties of InP NWs were experimentally verified by using them as a saturable absorber for a passively Q-switched Nd:YVO4 solid-state laser at 1.06 μm, where the shortest pulse width of 462 ns and largest single pulse energy of 1.32 μJ were obtained. Moreover, the ultrafast carrier relaxation dynamics were basically studied, and the intra-band and inter-band ultrafast carrier relaxation times of 8.1 and 63.8 ps, respectively, were measured by a degenerate pump–probe method with the probe laser of 800 nm. These results well demonstrate the nonlinear optical absorption properties, which show the excellent light manipulating capabilities of InP NWs and pave a way for their applications in ultrafast nanophotonic devices.