As the development of atom scale devices transitions from novel, proof-of-concept demonstrations to state-of-the-art commercial applications, automated assembly of such devices must be implemented. Here we present an automation method for the identification of defects prior to atomic fabrication via hydrogen lithography using deep learning. We trained a convolutional neural network to locate and differentiate between surface features of the technologically relevant hydrogen-terminated silicon surface imaged using a scanning tunneling microscope. Once the positions and types of surface features are determined, the predefined atomic structures are patterned in a defect-free area. By training the network to differentiate between common defects we are able to avoid charged defects as well as edges of the patterning terraces. Augmentation with previously developed autonomous tip shaping and patterning modules allows for atomic scale lithography with minimal user intervention.