Background: The mutational processes underlying non-coding cancer mutations and their biological significance in tumor evolution are poorly understood. To get better insights into the biological mechanisms of mutational processes in breast cancer, we integrate whole-genome level somatic mutations from breast cancer patients with chromatin states and transcription factor binding events. Results: We discover that a large fraction of non-coding somatic mutations in Estrogen Receptor (ER) positive breast cancers are confined to ER binding sites. Notably, the highly mutated estrogen receptor binding sites are associated with more frequent chromatin loop contacts and the associated distal genes are expressed at higher level. To elucidate the functional significance of these non-coding mutations, we focus on two of the recurrently mutated estrogen receptor binding sites. Our bioinformatics and biochemical analysis suggest loss of DNA-protein interactions due to the recurrent mutations. Through CRISPR interference, we find that the recurrently mutated regulatory element at the LRRC3C-GSDMA locus impacts the expression of multiple distal genes. Using a CRISPR base editor, we show that the recurrent C→T conversion at the ZNF143 locus results in decreased TF binding, increased chromatin loop formation, and increased expression of multiple distal genes. This single point mutation mediates reduced response to estradiol-induced cell proliferation but increased resistance to tamoxifen-induced growth inhibition. Conclusions: Our data suggest that ER binding is associated with localized accumulation of somatic mutations, some of which affect chromatin architecture, distal gene expression and cellular phenotypes in ER positive breast cancer. The dataset represents the source code associated with this work.