It is commonly accepted that androgen-producing fetal Leydig cells (FLC) are substituted by adult Leydig cells (ALC) during perinatal testis development. The mechanisms influencing this process are unclear. We used mice with a retinoid acid receptor 2 promoter-Cre recombinase transgene (Rarb-cre) expressed in embryonic FLC precursors, but not in postnatal testis, and a dual fluorescent Cre recombinase reporter to label FLC and ALC in vivo. All FLC in newborn testis had the recombinant, whereas the majority of LC in adult testis had the nonrecombinant reporter. Primary LC cultures from adult testis had either recombinant (20%) or nonrecombinant (80%) cells, demonstrating that the FLC survive in adult testis and their ontogeny is distinct from ALC. Conditional inactivation of androgen receptor (AR) allele using the Rarb-cre transgene resulted in a 50% increase of AR-negative LC in adult testis. The mutant males became infertile with age, with all LC in older testis showing signs of incomplete differentiation, such as a large number of big lipid droplets, an increase of finger-like protrusions, and a misexpression of steroidogenic or FLC- and ALC-specific genes. We propose that the antiandrogenic exposure during early development may similarly result in an increase of FLC in adult testis, leading to abnormal LC differentiation.