Polarized growth is maintained by both polarized exocytosis, which transports membrane components to specific locations on the cell cortex, and endocytosis, which retrieves these components before they can diffuse away. Despite functional links between these two transport pathways, they are generally considered to be separate events. Using live cell imaging, in vivo and in vitro protein binding assays, and in vitro pyrene-actin polymerization assays, we show that the yeast Rab GTPase Sec4p couples polarized exocytosis with cortical actin polymerization, which induces endocytosis. After polarized exocytosis to the plasma membrane, Sec4p binds Las17/Bee1p (yeast Wiskott-Aldrich Syndrome protein [WASp]) in a complex with Sla1p and Sla2p during actin patch assembly. Mutations that inactivate Sec4p, or its guanine nucleotide exchange factor (GEF) Sec2p, inhibit actin patch formation, whereas the activating sec4-Q79L mutation accelerates patch assembly. In vitro assays of Arp2/3-dependent actin polymerization established that GTPγS-Sec4p overrides Sla1p inhibition of Las17p-dependent actin nucleation. These results support a model in which Sec4p relocates along the plasma membrane from polarized sites of exocytic vesicle fusion to nascent sites of endocytosis. Activated Sec4p then promotes actin polymerization and triggers compensatory endocytosis, which controls surface expansion and kinetically refines cell polarization.