Regulation of chromatin structure is important for the control of DNA-templated processes such as gene expression and silencing, and its dysregulation is implicated in diverse developmental and cell proliferative defects such as tumorigenesis. Covalent post-translational modifications of histones are one of the prominent means to regulate the chromatin structure. Here, we summarize findings from our lab and others regarding the interactions between different covalent modifications of histones in the budding yeast Saccharomyces cerevisiae. First, we describe the effect of histone H3 phosphorylation at residue serine 10 in transcriptional gene activation, and its histone H3 acetylation dependent and independent modes of action and downstream effects on TATA-binding protein (TBP) recruitment. Further, we review how ubiquitylation of histone H2B and its deubiquitylation by ubiquitin proteases Ubp8 and Ubp10 regulate histone H3 methylations, and consequently affect co-activator-dependent gene transcription and silent chromatin, respectively.