Ferroelectric domain switching within individual nanoscale grains of a 100 nm thick polycrystalline PbZr0.2Ti0.8O3 thin film has been shown to depend on the relative crystallographic orientation of the adjacent grains. Using Piezoresponse Force Microscopy, the significance of local microstructure on the domain switching was demonstrated. Different regions within grains show different coercive fields under the same external electric field. In addition, neighboring grains also show a collective switching pattern, facilitating/suppressing switching on both sides of the grain boundaries compared to the center of the grain. These experimental observations were supported by numerical simulation demonstrating that changing the crystallographic orientation of a grain affects the switching loop of the neighboring grains. Based on both experimental and numerical simulation, the conclusion can be made that microstructural modulation of the local electric and stress field can significantly affect individual grain switching in polycrystalline thin films.