In yeast and humans, previous experiences can lead to epigenetic transcriptional memory: repressed genes that exhibit mitotically heritable changes in chromatin structure and promoter recruitment of poised RNA polymerase II preinitiation complex (RNAPII PIC), which enhances future reactivation. Here, we show that INO1 memory in yeast is initiated by binding of the Sfl1 transcription factor to the cis-acting Memory Recruitment Sequence, targeting INO1 to the nuclear periphery. Memory requires a remodeled form of the Set1/COMPASS methyltransferase lacking Spp1, which dimethylates histone H3 lysine 4 (H3K4me2). H3K4me2 recruits the SET3C complex, which plays an essential role in maintaining this mark. Finally, while active INO1 is associated with Cdk8- Mediator, during memory, Cdk8+ Mediator recruits poised RNAPII PIC lacking the Kin28 CTD kinase. Aspects of this mechanism are generalizable to yeast and conserved in human cells. Thus, COMPASS and Mediator are repurposed to promote epigenetic transcriptional poising by a highly conserved mechanism.