Observations show that 100 km class Kuiper Belt objects (KBOs) can be divided into (at least) two color groups, red (R; g - i < 1.2) and very red (VR; g - i > 1.2), reflecting a difference in their surface composition. This is thought to imply that KBOs formed over a relatively wide range of radial distance, r. The cold classicals at 42 au < r < 47 au are predominantly VR, and known Neptune Trojans at r ≃ 30 au are mostly R. Intriguingly, however, the dynamically hot KBOs show a mix of R and VR colors and no correlation of color with r. Here we perform migration/instability simulations where the Kuiper Belt is populated from an extended planetesimal disk. We find that the color observations can be best understood if R objects formed at r < r* and VR objects at r > r*, with 30 au < r* < 40 au. The proposed transition at 30 au < r* < 40 au would explain why the VR objects in the dynamically hot population have smaller orbital inclinations than the R objects, because the orbital excitation from Neptune weakens for orbits starting beyond 30 au. Possible causes of the R-VR color bimodality are discussed.