Geometrical frustration in spin systems often results in a large number of degenerate ground states. In this work, we study the antiferromagnetic Ising model on the three-dimensional swedenborgite lattice, which is a specific stacking of kagome and triangular layers. The model contains two exchange couplings, one within the kagome layer and another in between the kagome and triangular layers. We determine the phase diagram with and without an easy-axis magnetic field, and we calculate the ground-state degeneracy explicitly in terms of the residual entropy. At zero field, we find two different ground-state manifolds separated by a first-order transition at T =0 and equal exchange couplings. We also determine the T =0 phase diagram in a magnetic field and find a rich phase diagram with both degenerate and nondegenerate phases depending on the field strength and out-of-plane coupling.