This paper proposes a reconfigurable hardware accelerator design of five major high-order operators for vision sensor based robot systems. These five high-order operators include convolution, median filtering, Euclidean distance, vector inner-product and iToF, which are intensively used in robot vision algorithms. In this work, a reconfigurable hardware accelerator design method for multiple high-order operators is proposed. FPGA implementation results show that the proposed design has achieved area efficiency with 17.54% reduced LUTs and 44.02% reduced FFs against the baseline hardware design of the five high-order operators. Case studies of Laplace edge-detection and iToF benchmark demonstrate the energy efficiency of proposed design with 19.70% and 6.2% reduction in energy consumption, respectively.