The high rate of star formation and supernova explosions of starburst galaxies make them interesting sources of high-energy radiation. Depending on the level of turbulence present in their interstellar medium, the bulk of cosmic rays produced inside starburst galaxies may lose most of their energy before escaping, thereby making these sources behave as calorimeters, at least up to some maximum energy. Contrary to previous studies, here we investigate in detail the conditions under which cosmic ray confinement may be effective for electrons and nuclei and we study the implications of cosmic ray confinement in terms of multifrequency emission from starburst nuclei and production of high-energy neutrinos. The general predictions are then specialized to three cases of active starbursts, namely, M82, NGC 253, and Arp220. Both primary and secondary electrons, as well as electron-positron pairs produced by gamma-ray absorption inside starburst galaxies are taken into account. Electrons and positrons produced as secondary products of hadronic interactions are found to be responsible for most of the emission of leptonic origin. In particular, synchrotron emission of very high energy secondary electrons produces an extended emission of hard X-rays that represents a very interesting signature of hadronic process in starburst galaxies, potentially accessible to current and future observations in the X-ray band. A careful understanding of both the production and absorption of gamma-rays in starburst galaxies is instrumental to the assessment of the role of these astrophysical sources as sources of high-energy astrophysical neutrinos.