Continuing on our previous work [ArXiv:1212.2644], we develop semi-implicit numerical methods for solving low Mach number fluctuating hydrodynamic equations appropriate for modeling diffusive mixing in isothermal mixtures of fluids with different densities and transport coefficients. We treat viscous dissipation implicitly using a recently-developed variable-coefficient Stokes solver [ArXiv:1308.4605]. This allows us to increase the time step size significantly compared to the earlier explicit temporal integrator. For viscous-dominated flows, such as flows at small scales, we develop a scheme for integrating the overdamped limit of the low Mach equations, in which inertia vanishes and the fluid motion can be described by a steady Stokes equation. We also describe how to incorporate advanced higher-order Godunov advection schemes in the numerical method, allowing for the treatment of fluids with high Schmidt number including the vanishing mass diffusion coefficient limit. We incorporate thermal fluctuations in the description in both the inertial and overdamped regimes. We apply our algorithms to model the development of giant concentration fluctuations during the diffusive mixing of water and glycerol, and compare numerical results with experimental measurements. We find good agreement between the two, and observe propagative (non-diffusive) modes at small wavenumbers (large spatial scales), not reported in published experimental measurements of concentration fluctuations in fluid mixtures. Our work forms the foundation for developing low Mach number fluctuating hydrodynamics methods for miscible multi-species mixtures of chemically reacting fluids.