Quantum key distribution (QKD) protocols with threshold detectors are driving high-performance QKD demonstrations. The corresponding security proofs usually assume that all physical detectors have the same detection efficiency. However, the efficiencies of the detectors used in practice might show a mismatch depending on the manufacturing and setup of these detectors. A mismatch can also be induced as the different spatial-temporal modes of an incoming single might couple differently to a detector. Here we develop a method that allows to provide security proofs without the usual assumption. Our method can take the detection-efficiency mismatch into account without having to restrict the attack strategy of the adversary. Especially, we do not rely on any photon-number cut-off of incoming signals such that our security proof is complete. Though we consider polarization encoding in the demonstration of our method, the method applies to a variety of coding mechanisms, including time-bin encoding, and also allows for general manipulations of the spatial-temporal modes by the adversary. We thus can close the long-standing question how to provide a valid, complete security proof of a QKD setup with characterized efficiency mismatch. Our method also shows that in the absence of efficiency mismatch, the key rate increases if the loss due to detection inefficiency is assumed to be outside of the adversary's control, as compared to the view where for a security proof this loss is attributed to the action of the adversary.