In this paper, a hybrid optical-digital system is presented to solve a set of linear equations in iterative fashion. The optical part of the system consisting of a single holographic mask carries out the vector-matrix multiplication and the other part of the system consisting of a CCD detector and a microcomputer performs the measurement and addition of vectors. By using this system, a set of 4-dimensional equations are iteratively solved, and the experimental results show that the error of the solution is about 5% comparing with theoretical one.