Abstract Background Pluripotency of embryonic stem (ES) cells is controlled in part by chromatin-modifying factors that regulate histone H3 lysine 4 (H3K4) methylation. However, it remains unclear how H3K4 demethylation contributes to ES cell function. Results Here, we show that KDM5B, which demethylates lysine 4 of histone H3, co-localizes with H3K4me3 near promoters and enhancers of active genes in ES cells; its depletion leads to spreading of H3K4 methylation into gene bodies and enhancer shores, indicating that KDM5B functions to focus H3K4 methylation at promoters and enhancers. Spreading of H3K4 methylation to gene bodies and enhancer shores is linked to defects in gene expression programs and enhancer activity, respectively, during self-renewal and differentiation of KDM5B-depleted ES cells. KDM5B critically regulates H3K4 methylation at bivalent genes during differentiation in the absence of LIF or Oct4. We also show that KDM5B and LSD1, another H3K4 demethylase, co-regulate H3K4 methylation at active promoters but they retain distinct roles in demethylating gene body regions and bivalent genes. Conclusions Our results provide global and functional insight into the role of KDM5B in regulating H3K4 methylation marks near promoters, gene bodies, and enhancers in ES cells and during differentiation.