Bacteria in bulk fluids swim collectively and display fascinating emergent dynamics. Although bacterial collective swimming in three-dimensional (3D) geometries has been well studied, its counterpart in confined two-dimensional (2D) geometries relevant to natural habitats of bacteria is still poorly understood. Here, through carefully designed experiments on Escherichia coli in Hele-Shaw chambers, we show that a small change in the degree of confinement leads to a drastic change in bacterial collective swimming. While long-range nematic order emerges for bacteria that can cross during encounters, a slight decrease of the chamber height prevents the crossing, leading to the formation of bacterial clusters with short-range polar order. By tracking the swimming kinetics of individual bacteria, we reveal the microscopic origins of the two collective phases. Our study provides important insights into bacterial collective swimming under confinement and demonstrates a convenient way to control the emergent symmetry of collective phases.