We report suppression of superconductivity in Bi4O4S3 compound by Se doping at S site. Bulk polycrystalline samples are synthesized by solid state reaction route. The Rietveld refined of XRD data of all the studied samples show that the same are crystallized in tetragonal I4/mmm space group with slightly increased c lattice constant with the Se doping. Superconductivity is observed at below 4.5 K in both dc magnetization and resistivity with temperature measurements for all Bi4O4S3-xSex (x = 0, 0.03, 0.09 and 0.15) samples, though the same is decreased slightly with increase in Se content. The upper critical field of the Bi4O4S3 is estimated from resistivity under magnetic field [ρ(T)H] measurements of up to 2 T (Tesla), and the same decreases for Se doped samples. The flux flow activation energies being obtained by fitting to the Arrhenius equation are 18.60 meV for Bi4O4S3, 13.69 meV for Bi4O4S2.97Se0.03 and 13.16 meV for Bi4O4S2.85Se0.15 samples in applied magnetic field of 200 Oe. In conclusion the S site Se substitution showed detrimental effect on superconductivity of Bi4O4S3.