This article provides a tool for analyzing mechanisms that aim to achieve resilience against stealthy, or undetectable, attacks on cyber-physical systems (CPSs). We consider attackers who are able to corrupt all of the inputs and outputs of the system. To counter such attackers, a response scheme must be implemented that keeps the attacker from corrupting the inputs and outputs of the system for certain periods of time. To aid in the design of such a response scheme, our tool provides sufficient lengths for these periods of time in order to ensure safety with a particular probability. We provide a conservative upper bound on how long the system can remain under stealthy attack before the safety constraints are violated. Furthermore, we show how a detector limits the set of biases an attacker can exert on the system while still remaining stealthy, aiding a system operator in the design of the detector. Our contributions are demonstrated with an illustrative example.