Recently, reference-frame-independent quantum key distribution (RFI-QKD) has attracted more attention, since no active calibration of the reference frame is required. Thus, it has potential advantages in practical applications, especially for QKD with a mobile-platform-based quantum channel. However, due to the imperfections of devices, practical security is still one of the main bottlenecks for the application of RFI-QKD. Although, based on "qubit" assumption, RFI-QKD with the inaccuracy of encoded state was analyzed and demonstrated, this assumption is hard to be guaranteed in practical situations, and some source flaws could not be considered. In this paper we evaluate the security of RFI-QKD with general source flaws, in which the qubit assumption is removed. Then, almost all the source flaws (not only the inaccuracy of encoded state, but also the side channel, the Trojan horse, and so on) can be considered together.