The Finite Element Method (FEM) is widely used to solve discrete Partial Differential Equations (PDEs) in engineering applications. The popularity of FEM led to the development of a large family of variants, and, while their theoretical properties (such as convergence rate, stability, etc.) are understood well, their practical performance have not been systematically studied for large collections of automatically meshed 3D geometry. We introduce a set of benchmark problems, starting from simple cases with an analytical solution, moving to commonly used test problem setups, and finally fabricating solutions for thousands of real-world, automatically meshed geometries. For all these cases, we use state-of-the-art meshing tools to create both unstructured (tetrahedral) and structured (hexahedral) meshes, and compare the performance of different element types for a wide spectrum of elliptic PDEs ranging from heat diffusion to fluid propagation. We observe that, while linear tetrahedral elements perform poorly, often leading to locking artefacts, quadratic tetrahedral elements outperform hexahedral elements in all settings we tested. This observation suggests that for most problems in static structural analysis, thermal analysis, and low reynolds number flows, it is unnecessary to target automatic hex mesh generation, since high-quality results can be obtained with unstructured meshes, which can be created robustly and automatically with existing meshing algorithms. We released the description of the benchmark problems, meshes, and reference implementation of our testing infrastructure. This enables statistically significant comparisons between different FE methods, which we believe will provide a guide in the development of new meshing and FEA techniques.