Relocation of euchromatic genes near the heterochromatin region often results in mosaic gene silencing. In Saccharomyces cerevisiae, cells with the genes inserted at telomeric heterochromatin-like regions show a phenotypic variegation known as the telomere-position effect, and the epigenetic states are stably passed on to following generations. Here we show that the epigenetic states of the telomere gene are not stably inherited in cells either bearing a mutation in a catalytic subunit (Pol2) of replicative DNA polymerase epsilon (Pol epsilon) or lacking one of the nonessential and histone fold motif-containing subunits of Pol epsilon, Dpb3 and Dpb4. We also report a novel and putative chromatin-remodeling complex, ISW2/yCHRAC, that contains Isw2, Itc1, Dpb3-like subunit (Dls1), and Dpb4. Using the single-cell method developed in this study, we demonstrate that without Pol epsilon and ISW2/yCHRAC, the epigenetic states of the telomere are frequently switched. Furthermore, we reveal that Pol epsilon and ISW2/yCHRAC function independently: Pol epsilon operates for the stable inheritance of a silent state, while ISW2/yCHRAC works for that of an expressed state. We therefore propose that inheritance of specific epigenetic states of a telomere requires at least two counteracting regulators.