Nowadays, full face synthesis and partial face manipulation by virtue of the generative adversarial networks (GANs) have raised wide public concerns. In the multi-media forensics area, detecting and ultimately locating the image forgery have become imperative. We investigated the architecture of existing GAN-based face manipulation methods and observed that the imperfection of upsampling methods therewithin could be served as an important asset for GAN-synthesized fake images detection and forgery localization. Based on this basic observation, we have proposed a novel approach to obtain high localization accuracy, at full resolution, on manipulated facial images. To the best of our knowledge, this is the very first attempt to solve the GAN-based fake localization problem with a gray-scale fakeness prediction map that preserves more information of fake regions. To improve the universality of FakeLocator across multifarious facial attributes, we introduce an attention mechanism to guide the training of the model. Experimental results on the CelebA and FFHQ databases with seven different state-of-the-art GAN-based face generation methods show the effectiveness of our method. Compared with the baseline, our method performs two times better on various metrics. Moreover, the proposed method is robust against various real-world facial image degradations such as JPEG compression, low-resolution, noise, and blur.