LSD2/AOF1/KDM1b catalyzes demethylation of mono- and di-methylated H3K4 and plays an important role in transcriptional regulation and genomic imprinting. Here, we report the high-resolution crystal structures of apo-LSD2 and LSD2 in complex with a peptide that mimics H3K4me2. Three structural domains of LSD2, namely, the novel N-terminal zinc finger, the centrally located SWIRM domain, and the C-terminal oxidase domain, closely pack together to form a boot-shaped structure. The active site cavity in the oxidase domain is large enough to accommodate several residues of the histone H3 tail and cannot discriminate between the different states of H3K4 methylation. The N-terminal zinc-finger domain, composed of a novel C4H2C2-type zinc finger and a specific CW-type zinc finger, is required for demethylase activity and, surprisingly, the binding of cofactor flavin adenine dinucleotide (FAD). In fact, a relay of extensive interactions through the zinc finger-SWIRM-oxidase domains is required for LSD2 demethylase activity and the binding of FAD. These results reveal a novel mechanism for the zinc finger and SWIRM domains in controlling LSD2 demethylase activity and provide a framework for elucidating the regulation and function of LSD2.