Sterols are crucial components of biological membranes, which are synthetized in the ER and accumulate in the plasma membrane (PM). Here, by applying a genetically encoded sterol biosensor (D4H), we visualize a sterol flow between PM and endosomes in the fission yeast Schizosaccharomyces pombe. Using time-lapse and correlative light-electron microscopy, we found that inhibition of Arp2/3-dependent F-actin assembly promotes the reversible relocalization of D4H from the PM to internal sterol-rich compartments (STRIC) labeled by synaptobrevin Syb1. Retrograde sterol internalization to STRIC is independent of endocytosis or an intact Golgi, but depends on Ltc1, a LAM/StARkin-family protein localized to ER-PM contact sites. The PM in ltc1Δ cells over-accumulates sterols and upon Arp2/3 inhibition forms extended ER-interacting invaginations, indicating that sterol transfer contributes to PM size homeostasis. Anterograde sterol movement from STRIC is independent of canonical vesicular trafficking but requires Arp2/3, suggesting a novel role for this complex. Thus, transfer routes orthogonal to vesicular trafficking govern the flow of sterols in the cell.