Counterpropagating (CP) solitons generated in high-Q microcavities not only offer useful dual-comb sources, but also provide a new platform to study soliton interactions. Here, we demonstrate and theoretically explain a manifestation of soliton trapping that occurs between CP solitons in a silica microcavity introducing a Kerr soliton dimer. In conventional soliton trapping, the group velocities of two solitons can be synchronized by a Kerr-effect-mediated interaction. The solitons can then copropagate with a fixed temporal delay. However, as shown here, when counterpumping a microcavity using slightly detuned pump frequencies and in the presence of backscattering, the group velocities of clockwise and counterclockwise solitons undergo periodic modulation instead of being locked to a constant velocity. Upon emission from the microcavity, the solitons feature a relative oscillatory motion around a locked average relative displacement with an amplitude that can be larger than the soliton pulse width. This relative motion introduces a sideband fine structure into the optical spectrum of the CP solitons. Our observation provides insights on coherently pumped soliton dimers in microcavities.