Duality of the conformal blocks of a rational conformal field theory defines matrices which may be used to construct representations of all monodromies and modular transformations in the theory. These duality matrices satisfy a finite number of independent polynomial equations, which imply constraints on monodromies allowed in rational conformal field theories. The equations include a key identity needed to prove a recent conjecture of Verlinde that the one-loop modular transformation S diagonalizes the fusion rules. Using this formalism we show that duality of the g=0 four-point function and modular invariance of all one-loop one-point functions guarantee modular invariance to all orders. The equations for duality matrices should be useful in the classification of conformal field theories.