Symmetry-protected topological (SPT) phases are gapped phases of matter that cannot be deformed to a trivial phase without breaking the symmetry or closing the bulk gap. Here we introduce a notion of a topological obstruction that is not captured by bulk energy gap closings in periodic boundary conditions. More specifically, given a symmetric boundary termination we say two bulk Hamiltonians belong to distinct boundary obstructed topological phases (BOTPs) if they can be deformed to each other on a system with periodic boundaries, but cannot be deformed to each other in the open system without closing the gap at at least one high-symmetry surface. BOTPs are not topological phases of matter in the standard sense since they are adiabatically deformable to each other on a torus, but, similar to SPTs, they are associated with boundary signatures in open systems such as surface states or fractional corner charges. In contrast to SPTs, these boundary signatures are not anomalous and can be removed by symmetrically adding lower-dimensional SPTs on the boundary, but they are stable as long as the spectral gap at high-symmetry edges/surfaces remains open. We show that the double-mirror quadrupole model of [W. A. Benalcazar, B. A. Bernevig, and T. L. Hughes, Science 357, 61 (2017), 10.1126/science.aah6442] is a prototypical example of such phases, and present a detailed analysis of several aspects of boundary obstructions in this model. In addition, we introduce several three-dimensional models having boundary obstructions, which are characterized either by surface states or fractional corner charges. Furthermore, we provide a complete characterization of boundary obstructed phases in terms of symmetry representations. Namely, two distinct BOTP phases correspond to equivalent band representations in the periodic system which become inequivalent upon restricting the symmetry group to that of the open system. This is used to shown that for a given open boundary, there is only one class of BOTPs which corresponds to a local representation of the symmetry of the open system and thus can be designated as the trivial phase. All other BOTP classes do not correspond to local representation of the open system and as a result necessarily exhibit a filling anomaly or gapless surface states.