Vector structured beams (VSBs) offer infinite eigenstates and open up new possibilities for high-capacity optical and quantum communications by the multiplexing of the states. Therefore, the sorting and measuring of VSBs are extremely important. However, the efficient manipulations of a large number of VSBs have simultaneously remained challenging up to now, especially in integrated optical systems. Here, we propose a compact spin-multiplexed diffractive metasurface capable of continuously sorting and detecting arbitrary VSBs through spatial intensity separation. By introducing a diffractive optical neural network with cascaded metasurface systems, we demonstrate arbitrary VSBs sorters that can simultaneously identify Laguerre–Gaussian modes (l = - 4 to 4, p = 1 to 4), Hermitian–Gaussian modes (m = 1 to 4, n = 1 to 3), and Bessel–Gaussian modes (l = 1 to 12). Such a sorter for arbitrary VSBs could revolutionize applications in integrated and high-dimensional optical communication systems.