Spermatogenesis, a fundamental process in the male reproductive system, requires a series of tightly controlled epigenetic and genetic events in germ cells ranging from spermatogonia to spermatozoa. Jmjd1a is a key epigenetic regulator expressed in the testis. It specifically demethylates mono- and di-methylated histone H3 lysine 9 (H3K9me1 and H3K9me2) but not tri-methylated H3K9 (H3K9me3). In this study, we generated a Jmjd1a antibody for immunohistochemistry and found Jmjd1a was specifically produced in pachytene and secondary spermatocytes. Disruption of the Jmjd1a gene in mice significantly increased H3K9me1 and H3K9me2 levels in pachytene spermatocytes and early elongating spermatids without affecting H3K9me3 levels. Concurrently, the levels of histone acetylation were decreased in Jmjd1a knock-out germ cells. This suggests Jmjd1a promotes transcriptional activation by lowering histone methylation and increasing histone acetylation. Interestingly, the altered histone modifications in Jmjd1a-deficient germ cells caused diminished cAMP-response element modulator (Crem) recruitment to chromatin and decreased expression of the Crem coactivator Act and their target genes Tnp1 (transition protein 1), Tnp2, Prm1 (protamine 1), and Prm2, all of which are essential for chromatin condensation in spermatids. In agreement with these findings, Jmjd1a deficiency caused extensive germ cell apoptosis and blocked spermatid elongation, resulting in severe oligozoospermia, small testes, and infertility in male mice. These results indicate that the Jmjd1a-controlled epigenetic histone modifications are crucial for Crem-regulated gene expression and spermatogenesis.