Highly prolate-shaped whispering-gallery-mode "bottle microresonators" have recently attracted considerable attention due to their advantageous properties. We experimentally show that such resonators offer ultra-high quality factors, microscopic mode volumes, and near lossless in- and out-coupling of light using ultra-thin optical fibers. Additionally, bottle microresonators have a simple and customizable mode structure. This enables full tunability using mechanical strain and simultaneous coupling of two ultra-thin coupling fibers in an add-drop configuration. We present two applications based on these characteristics: In a cavity quantum electrodynamics experiment, we actively stabilize the frequency of the bottle microresonator to an atomic transition and operate it in an ultra-high vacuum environment in order to couple single laser-cooled atoms to the resonator mode. In a second experiment, we show that the bottle microresonator can be used as a low-loss, narrow-band add-drop filter. Using the Kerr effect of the silica resonator material, we furthermore demonstrate that this device can be used for single-wavelength all-optical signal processing.