A convex sequential effect algebra (COSEA) is an algebraic system with three physically motivated operations, an orthogonal sum, a scalar product and a sequential product. The elements of a COSEA correspond to yes-no measurements and are called effects. In this work we stress the importance of contexts in a COSEA. A context is a finest sharp measurement and an effect will act differently according to the underlying context with which it is measured. Under a change of context, the possible values of an effect do not change but the way these values are obtained may be different. In this paper we discuss direct sums and the center of a COSEA. We also consider conditional probabilities and the spectra of effects. Finally, we characterize COSEA's that are isomorphic to COSEA's of positive operators on a complex Hilbert space. These result in the traditional quantum formalism. All of this work depends heavily on the concept of a context.