Recently, machine learning has been applied successfully for identifying phases and phase transitions of the Ising models. The continuous phase transition is characterized by spontaneous symmetry breaking, which can not be detected in general from a single spin configuration. To investigate if neural networks can extract correlations among spin snapshots, we propose a new test using the random-gauge Ising model. We show that neural networks can extract the order parameter or the energy of the random-gauge model as in the ferromagnetic case. We also discuss how and where the information of random gauge is encoded in neural networks and attempt to reconstruct the gauge from the neural network parameters. We find that the fully connected network encodes the effect of random gauge to its weights naturally. In contrast, the convolutional network copes with the randomness by assigning different network parts to local gauge patterns. This observation indicates that although the latter demonstrates higher performance than the former for the present randomized-gauge test, the former is more effective and suitable for dealing with models with spatial randomness.