Although Gödel's incompleteness theorem made mathematician recognize that no axiomatic system could completely prove its correctness and that there is an eternal hole between our knowledge and the world, physicists so far continue to work on the approaches based on the hypothesis to completely or approximately know the systems of interest. In this paper, however, I review the recent development of a different approach, a statistical theory based upon the notion of incomplete information. This consideration leads to generalized statistical mechanics characterized by an incompleteness parameter which equals unity when information is complete. The mathematical and physical bases of the information incompleteness are discussed. The application of the concomitant incomplete quantum distribution to correlated electron systems is reviewed.