Van Daele and Wang developed a purely algebraic notion of weak multiplier Hopf algebras, which extends the notions of Hopf algebras, multiplier Hopf algebras, and weak Hopf algebras. With an additional requirement of an existence of left or right integrals, this framework provides a self-dual class of algebraic quantum groupoids. The aim of this paper is to show that from this purely algebraic data, with only a minimal additional requirement ("quasi-invariance"), one can construct a $C^*$-algebraic quantum groupoid of separable type, recently defined by the author, with Van Daele. The $C^*$-algebraic quantum groupoid is represented as an operator algebra on the Hilbert space constructed from the left integral, and the comultiplication is determined by means of a certain multiplicative partial isometry $W$, which is no longer unitary. In the last section (Appendix), we obtain some results in the purely algebraic setting, which have not appeared elsewhere.