Er3+ doped and Yb3+/Er3+ co-doped Y4Al2O9 phosphors are prepared by the sol-gel method. The effect of dopant concentration on the structure and up-conversion properties is investigated by X-ray diffraction (XRD) and photoluminescence, respectively. XRD pattern indicates that the sample structure belongs to monoclinic. Under 980 nm excitation, the green and red up-conversion emissions are observed and the emission intensities depended on the Yb3+ ion concentration. The green up-conversion emissions decrease with the increase of Yb3+ concentration, while red emission increases as Yb3+ concentration increases from 0 to 8 at% and then decreases at high Yb3+ concentration. The mechanisms of the up-conversion emissions are discussed and results shows that in Er3+ and Yb3+/Er3+ co-doped system, cross-relaxation (CR) and energy transfer (ET) processes play an important role for the green and red up-conversion emissions.