We consider a topological quiver matrix model which is expected to give a dual description of the instanton dynamics of topological U(N) gauge theory on D6 branes. The model is a higher dimensional analogue of the ADHM matrix model that leads to Nekrasov's partition function. The fixed points of the toric action on the moduli space are labeled by colored plane partitions. Assuming the localization theorem, we compute the partition function as an equivariant index. It turns out that the partition function does not depend on the vacuum expectation values of Higgs fields that break U(N) symmetry to U(1)N at low energy. We conjecture a general formula of the partition function, which reduces to a power of the MacMahon function, if we impose the Calabi-Yau condition. For non Calabi-Yau case we prove the conjecture up to the third order in the instanton expansion.