Multiplexing technology serves as an effective approach to increase both information storage and transmission capability. However, when exploring multiplexing methods across various dimensions, the polarization dimension encounters limitations stemming from the finite orthogonal combinations. Given that only two mutually orthogonal polarizations are identifiable on the basic Poincaré sphere, this poses a hindrance to polarization modulation. To overcome this challenge, we propose a construction method for the optical polarized orthogonal matrix (OPOM), which is not constrained by the number of orthogonal combinations. Furthermore, we experimentally validate its application in high-dimensional multiplexing of polarization holography. We explore polarization holography technology, capable of recording amplitude, phase, and polarization, for the purpose of recording and selective reconstruction of polarization multi-channels. Our research reveals that, despite identical polarization states, multiple images can be independently manipulated within distinct polarization channels through orthogonal polarization combinations, owing to the orthogonal selectivity among information. By selecting the desired combination of input polarization states, the reconstructed image can be switched with negligible crosstalk. This non-square matrix composed of polarization unit vectors provides prospects for multi-channel information retrieval and dynamic display, with potential applications in optical communication, optical storage, logic devices, anti-counterfeiting, and optical encryption.