The study of low-dimensional quantum systems has proven to be a particularly fertile field for discovering novel types of quantum matter. When studied numerically, low-energy states of low-dimensional quantum systems are often approximated via a tensor-network description. The tensor network's utility in studying short range correlated states in 1D have been thoroughly investigated, with numerous examples where the treatment is essentially exact. Yet, despite the large number of works investigating these networks and their relations to physical models, examples of exact correspondence between the ground state of a quantum critical system and an appropriate scale-invariant tensor network have eluded us so far. Here we show that the features of the quantum-critical Motzkin model can be faithfully captured by an analytic tensor network that exactly represents the ground state of the physical Hamiltonian. In particular, our network offers a two-dimensional representation of this state by a correspondence between walks and a type of tiling of a square lattice. We discuss connections to renormalization and holography.