Restart—interrupting a stochastic process followed by a new start—is known to improve the mean time to its completion, and the general conditions under which such an improvement is achieved are now well understood. Here, we explore how restart affects other important metrics of first-passage phenomena, namely, the median and the mode of the first-passage time distribution. Our analysis provides a general criterion for when restart lowers the median time and demonstrates that restarting is always helpful in reducing the mode. Additionally, we show that simple nonuniform restart strategies allow to optimize the mean and the median first-passage times, regardless of the characteristic timescales of the underlying process. These findings are illustrated with the canonical example of a diffusive search with resetting.