We construct numerical approximations for Mean Field Games with fractional or nonlocal diffusions. The schemes are based on semi-Lagrangian approximations of the underlying control problems/games along with dual approximations of the distributions of agents. The methods are monotone, stable, and consistent, and we prove convergence along subsequences for (i) degenerate equations in one space dimension and (ii) nondegenerate equations in arbitrary dimensions. We also give results on full convergence and convergence to classical solutions. Numerical tests are implemented for a range of different nonlocal diffusions and support our analytical findings.