The domain of high speed optical astrophysics is still quite unexplored. The availability of 10 meter diameter telescopes offers the unique possibility to investigate variability of faint objects at submillisecond time scales. In this paper I describe the concepts of a photometer and a spectrometer for high speed astronomical observations. The instruments are based on a photon counting detector developed for high energy physics, the Visible Light Photon Counter (VLPC). The detector has a quantum efficiency in the visible as high as 88% and performs photon counting with sub microsecond time resolution. The photometer is built using VLPC arrays. Adding a grating a VLPC array can be used in a time resolved spectrograph with medium resolution. This paper develops, starting from experimental data, the concept of the two VLPC based instruments and their application to time resolved photometry and spectroscopy of compact objects (pulsars, cataclysmic variables, low mass X-ray binary systems etc) and optical counterparts of Gamma Ray Bursts. The high speed optical observations are the ideal complement to X/γ rays and gravitational wave studies. The application of the instruments to the optical photometry of pulsars, the spectrophotometry of the prompt optical flash from Gamma Ray Bursts and the study of binary systems are discussed in detail: in the last two applications the instruments offer better opportunities than existing instruments.