Carbon nanotube (CNT) film nanobolometers take advantages of high infrared absorption of CNTs, proving a promising alternative for low-cost, uncooled infrared detection. The performance of the CNT nanobolometers is determined by the optoelectronic process on CNTs at a microscopic scale, which links intimately to the diameter of the CNT—a critical parameter that intrinsically affects the band gap and hence infrared absorption, as well as extrinsically affects the surface oxygen adsorption effect and thermal-link of the CNT detector element to the environment. Both the intrinsic and extrinsic factors play important roles in the photoresponse, noise spectrum and the figure-of-merit detectivity D* of the CNT nanobolometers and their interplay determines the device’s ultimate performance. In this work, we present a systematic study of the effect of CNT diameter in the range of 1-50 nm on the physical properties relevant to CNT nanobolometers. The optimal CNT diameter was found to be in the range of 2-12 nm with the D* up to 3.3 × 107 cm(Hz)1/2 W-1, which represents an order of magnitude improvement over the best D* reported previously on CNT film nanobolometers.