A fast system to reveal the presence and type of fabric defects during the weaving process is developed. Since the fabric is similar to a 2D grid, its defects are clearly observed in the changes in its optical Fourier transform (OFT), which appears stationary while the fabric is moving across the loom. Previous work, based on the statistical parameters of the OFT, showed that the presence of faults can be detected when only global changes in the images are considered. We show that by selecting a small subset of pixels from the image as input to a neural network, fabric defects can not only be detected but also successfully identified. A knowledge-based system could conceivably be constructed to use this information to resolve problems with the loom in real time, without the need for operator intervention.