Owing to the ability to parallel manipulate micro-objects, dynamic holographic optical tweezers (HOTs) are widely used for assembly and patterning of particles or cells. However, for simultaneous control of large-scale targets, potential collisions could lead to defects in the formed patterns. Herein we introduce the artificial potential field (APF) to develop dynamic HOTs that enable collision-avoidance micro-manipulation. By eliminating collision risks among particles, this method can maximize the degree of parallelism in multi-particle transport, and it permits the implementation of the Hungarian algorithm for matching the particles with their target sites in a minimal pathway. In proof-of-concept experiments, we employ APF-empowered dynamic HOTs to achieve direct assembly of a defect-free 8 × 8 array of microbeads, which starts from random initial positions. We further demonstrate successive flexible transformations of a 7 × 7 microbead array, by regulating its tilt angle and inter-particle spacing distances with a minimalist path. We anticipate that the proposed method will become a versatile tool to open up new possibilities for parallel optical micromanipulation tasks in a variety of fields.