It is shown that the classical L-operator algebra of the elliptic Ruijsenaars-Schneider model can be realized as a subalgebra of the algebra of functions on the cotangent bundle over the centrally extended current group in two dimensions. It is governed by two dynamical r and ?r-matrices satisfying a closed system of equations. The corresponding quantum R and ?R-matrices are found as solutions to quantum analogs of these equations. We present the quantum L-operator algebra and show that the system of equations on R and ?R arises as the compatibility condition for this algebra. It turns out that the R-matrix is twist-equivalent to the Felder elliptic RF-matrix with ?R playing the role of the twist. The simplest representation of the quantum L-operator algebra corresponding to the elliptic Ruijsenaars-Schneider model is obtained. The connection of the quantum L-operator algebra to the fundamental relation RLL=LLR with Belavin's elliptic R matrix is established. As a byproduct of our construction, we find a new N-parameter elliptic solution to the classical Yang-Baxter equation.