We study the evolution of genealogies of a population of individuals, whose type frequencies result in an interacting Fleming-Viot process on $\Z$. We construct and analyze the genealogical structure of the population in this genealogy-valued Fleming-Viot process as a marked metric measure space, with each individual carrying its spatial location as a mark. We then show that its time evolution converges to that of the genealogy of a continuum-sites stepping stone model on $\R$, if space and time are scaled diffusively. We construct the genealogies of the continuum-sites stepping stone model as functionals of the Brownian web, and furthermore, we show that its evolution solves a martingale problem. The generator for the continuum-sites stepping stone model has a singular feature: at each time, the resampling of genealogies only affects a set of individuals of measure $0$. Along the way, we prove some negative correlation inequalities for coalescing Brownian motions, as well as extend the theory of marked metric measure spaces (developed recently by Depperschmidt, Greven and Pfaffelhuber [DGP12]) from the case of probability measures to measures that are finite on bounded sets.