In condensed-matter systems, competition between ground states at phase boundaries can lead to significant changes in material properties under external stimuli, particularly when these ground states have different crystal symmetries. A key scientific and technological challenge is to stabilize and control coexistence of symmetry-distinct phases with external stimuli. Using BiFeO3 (BFO) layers confined between layers of the dielectric TbScO3 as a model system, we stabilize the mixed-phase coexistence of centrosymmetric and non-centrosymmetric BFO phases with antipolar, insulating and polar, semiconducting behavior, respectively at room temperature. Application of in-plane electric (polar) fields can both remove and introduce centrosymmetry from the system resulting in reversible, nonvolatile interconversion between the two phases. This interconversion between the centrosymmetric insulating and non-centrosymmetric semiconducting phases coincides with simultaneous changes in the non-linear optical response of over three orders of magnitude, a change in resistivity of over five orders of magnitude, and a change in the polar order. Our work establishes a materials platform allowing for novel cross-functional devices which take advantage of changes in optical, electrical, and ferroic responses.