In the earlier paper (Ch. Venkatesh et al. Solid State Comm., 2010), a dielectric percolation has been reported in composites of nano-quasicrystalline (nQc) Al-Cu-Fe (filler) and polyvinylidene fluoride (PVDF) (matrix). Though a high value of dielectric constant is observed near to the percolation threshold, these composites show higher dielectric loss values at lower frequencies. An effect of annealing has been investigated on the same composites which lead to decrement in dielectric loss values appreciably. The low dielectric loss values have been attributed due to the formation of thick grain boundaries in polymer matrix which completely surrounds the nQc cluster that weakens the effective tunneling of charge carriers near the filler-polymer interfacial region. Finally, a dielectric constant of 500 with tanδ<1 at 1 KHz is observed which may make the composite useful for low loss capacitive applications.