Classification of stars, by comparing their optical spectra to a few dozen spectral standards, has been a workhorse of observational astronomy for more than a century. Here, we extend this technique by compiling a library of optical spectra of 404 touchstone stars observed with Keck/HIRES by the California Planet Search. The spectra have high resolution (R ≈ 60,000), high signal-to-noise ratio (S/N ≈ 150/pixel), and are registered onto a common wavelength scale. The library stars have properties derived from interferometry, asteroseismology, LTE spectral synthesis, and spectrophotometry. To address a lack of well-characterized late-K dwarfs in the literature, we measure stellar radii and temperatures for 23 nearby K dwarfs, using modeling of the spectral energy distribution and Gaia parallaxes. This library represents a uniform data set spanning the spectral types ∼M5-F1 (T eff ≈ 3000-7000 K, R ⋆ ≈ 0.1-16 R ☉). We also present “Empirical SpecMatch” (SpecMatch-Emp), a tool for parameterizing unknown spectra by comparing them against our spectral library. For FGKM stars, SpecMatch-Emp achieves accuracies of 100 K in effective temperature (T eff), 15% in stellar radius (R ⋆), and 0.09 dex in metallicity ([Fe/H]). Because the code relies on empirical spectra it performs particularly well for stars ∼K4 and later, which are challenging to model with existing spectral synthesizers, reaching accuracies of 70 K in T eff, 10% in R ⋆, and 0.12 dex in [Fe/H]. We also validate the performance of SpecMatch-Emp, finding it to be robust at lower spectral resolution and S/N, enabling the characterization of faint late-type stars. Both the library and stellar characterization code are publicly available.