Based on the framework of the spin-flip model (SFM), the bistability performances of time-varying orthogonal optical injected vertical-cavity surface-emitting lasers (VCSEL) subject to weak optical feedback have been investigated theoretically. The results show that, for a slave VCSEL (S-VCSEL) subject to time-varying orthogonal optical injection from a master VCSEL (M-VCSEL), the introduction of weak optical feedback does not change the varied trend that hysteresis width increases with the increase of the sweeping rate of optical injection coefficient. For a fixed sweeping rate of the injection coefficient, with the increase of the feedback strength, the hysteresis width will narrow. Under the case that there exists frequency detuning Δν(=νm-νs, νm and νs are the free running frequencies of M-VCSEL and S-VCSEL, respectively) between the M-VCSEL and S-VCSEL, with the increase of Δν from negative value to positive value, the hysteresis width narrows firstly, and after reaching a minimum, widens finally. The obvious fluctuation of hysteresis width appearing in the negative frequency detuning region can be suppressed to some extent after introducing optical feedback.