The habenular neurons on both sides of the zebrafish diencephalon show an asymmetric (laterotopic) axonal projection pattern into the interpeduncular nucleus. We previously revealed that the habenula could be subdivided into medial and lateral subnuclei, and a prominent left-right difference in the size ratio of these subnuclei accounts for the asymmetry in its neural connectivity. In the present study, birth date analysis showed that neural precursors for the lateral subnuclei were born at earlier stages than those for the medial subnuclei. More neurons for the early-born lateral subnuclei were generated on the left side, while more neurons for the late-born medial subnuclei were generated on the right side. Genetic hyperactivation and repression of Notch signaling revealed that differential timing determines both specificity and asymmetry in the neurogenesis of neural precursors for the habenular subnuclei.