Coronal holes are the observational manifestation of the solar magnetic field open to the heliosphere and are of pivotal importance for our understanding of the origin and acceleration of the solar wind. Observations from space missions such as the Solar Dynamics Observatory now allow us to study coronal holes in unprecedented detail. Instrumental effects and other factors, however, pose a challenge to automatically detect coronal holes in solar imagery. The science community addresses these challenges with different detection schemes. Until now, little attention has been paid to assessing the disagreement between these schemes. In this COSPAR ISWAT initiative, we present a comparison of nine automated detection schemes widely-applied in solar and space science. We study, specifically, a prevailing coronal hole observed by the Atmospheric Imaging Assembly instrument on 2018 May 30. Our results indicate that the choice of detection scheme has a significant effect on the location of the coronal hole boundary. Physical properties in coronal holes such as the area, mean intensity, and mean magnetic field strength vary by a factor of up to 4.5 between the maximum and minimum values. We conclude that our findings are relevant for coronal hole research from the past decade, and are therefore of interest to the solar and space research community.