Nowadays unmanned aerial vehicles (UAVs) are being widely applied to a wealth of civil and military applications. Robust and high-throughput wireless communication is the crux of these UAV applications. Yet, air-to-ground links suffer from time-varying channels induced by the agile mobility and dynamic environments. Rate adaptation algorithms are generally used to choose the optimal data rate based on the current channel conditions. State-of-the-art approaches leverage physical layer information for rate adaptation, and they work well under certain conditions. However, the above protocols still have limitation under constantly changing flight states and environments for air-to-ground links. To solve this problem, we propose StateRate, a state-optimized rate adaptation algorithm that fully exploits the characteristics of UAV systems using a hybrid deep learning model. The key observation is that the rate adaptation strategy needs to be adjusted according to motion-dependent channel models, which can be reflected by flight states. In this work, the rate adaptation protocol is enhanced with the help of the on-board sensors in UAVs. To make full use of the sensor data, we introduce a learning-based prediction module by leveraging the internal state to dynamically store temporal features under variable flight states. We also present an online learning algorithm by employing the pre-trained model that adapts the rate adaptation algorithm to different environments. We implement our algorithm on a commercial UAV platform and evaluate it in various environments. The results demonstrate that our system outperforms the best-known rate adaptation algorithm up to 53% in terms of throughput when the velocity is 2-6~m/s.