GLP and G9a are major H3K9 dimethylases and are essential for mouse early embryonic development. GLP and G9a both harbor ankyrin repeat domains that are capable of binding H3K9 methylation. However, the functional significance of their recognition of H3K9 methylation is unknown. Here, we report that the histone methyltransferase activities of GLP and G9a are stimulated by neighboring nucleosomes that are premethylated at H3K9. These stimulation events function in cis and are dependent on the H3K9 methylation binding activities of ankyrin repeat domains of GLP and G9a. Disruption of the H3K9 methylation-binding activity of GLP in mice causes growth retardation of embryos, ossification defects of calvaria, and postnatal lethality due to starvation of the pups. In mouse embryonic stem cells (ESCs) harboring a mutant GLP that lacks H3K9me1-binding activity, critical pluripotent genes, including Oct4 and Nanog, display inefficient establishment of H3K9me2 and delayed gene silencing during differentiation. Collectively, our study reveals a new activation mechanism for GLP and G9a that plays an important role in ESC differentiation and mouse viability.