Accelerator performance often deteriorates with time during a long period of operation due to secular changes in the machine components or the surrounding environment. In many cases some tuning knobs are effective in compensating the performance drifts and optimization methods can be used to find the ideal machine setting. However, such intervention usually cannot be done without interrupting user operation as the optimization algorithms can substantially impact the machine performance. We propose an optimization algorithm, Safe Robust Conjugate Direction Search (RCDS-S), which can perform accelerator tuning while keeping the machine performance within a designated safe envelope. The algorithm builds probability models of the objective function using Lipschitz continuity of the function as well as characteristics of the drifts and applies to the selection of trial solutions to ensure the machine operates safely during tuning. The algorithm can run during normal user operation constantly, or periodically, to compensate the performance drifts. Simulation and online tests have been done to validate the performance of the algorithm.