We have synthesized the Fe1-xMnxV2O4 (0 ≦ x ≦ 1) system, and have investigated its structural and magnetic properties through synchrotron powder diffraction, magnetization and specific heat measurements. We have examined the local distortion for both the FeO4 tetrahedron and VO6 octahedron on the basis of structural refinements using the synchrotron diffraction data. We found that the behaviour of these local distortions and the temperature dependent magnetization changes continuously with respect to the Mn2+ concentration, x. The local distortion of the FeO4 tetrahedron derived from the orbital order of the x2-y2 type below the ferrimagnetic transition temperature becomes smaller for x ≦ 0.6, and is absent for x > 0.6. The ferro-orbital order, leading to compressed local distortion of VO6 below the non-collinear ferrimagnetic order temperature, is gradually suppressed with increasing x and changes to the antiferro-orbital order for x > 0.6, for which the long range orbital order of Fe2+ disappears. We suggest that the two types of V3+ orbital orders are strongly correlated with the non-collinear magnetic order, and that the ferro-orbital order is possibly stabilized by the orbital degrees of freedom of the Fe2+ ions located at the A-site.