We demonstrate that at long times the rate of passive scalar decay in a turbulent, or simply chaotic, flow is dominated by regions where mixing is less efficient. We examine two situations. The first is of a spatially homogeneous stationary turbulent flow with both viscous and inertial scales present. It is shown that at large times scalar fluctuations decay algebraically in time at all spatial scales. The second example explains chaotic stationary flow in a disk/pipe. The boundary region of the flow controls the long-time decay, which is algebraic at some transient times, but becomes exponential, with the decay rate dependent on the scalar diffusion coefficient, at longer times.