This paper introduces a Markov chain approach that allows a rigorous analysis of agent based opinion dynamics as well as other related agent based models (ABM). By viewing the ABM dynamics as a micro description of the process, we show how the corresponding macro description is obtained by a projection construction. Then, well known conditions for lumpability make it possible to establish the cases where the macro model is still Markov. In this case we obtain a complete picture of the dynamics including the transient stage, the most interesting phase in applications. For such a purpose a crucial role is played by the type of probability distribution used to implement the stochastic part of the model which defines the updating rule and governs the dynamics. In addition, we show how restrictions in communication leading to the co-existence of different opinions follow from the emergence of new absorbing states. We describe our analysis in detail with some specific models of opinion dynamics. Generalizations concerning different opinion representations as well as opinion models with other interaction mechanisms are also discussed. We find that our method may be an attractive alternative to mean-field approaches and that this approach provides new perspectives on the modeling of opinion exchange dynamics, and more generally of other ABM.