E-cadherin plays a pivotal role in epithelial morphogenesis. It controls the intercellular adhesion required for tissue cohesion and anchors the actomyosin-driven tension needed to change cell shape. In the early Drosophila embryo, Myosin-II (Myo-II) controls the planar polarized remodelling of cell junctions and tissue extension. The E-cadherin distribution is also planar polarized and complementary to the Myosin-II distribution. Here we show that E-cadherin polarity is controlled by the polarized regulation of clathrin- and dynamin-mediated endocytosis. Blocking E-cadherin endocytosis resulted in cell intercalation defects. We delineate a pathway that controls the initiation of E-cadherin endocytosis through the regulation of AP2 and clathrin coat recruitment by E-cadherin. This requires the concerted action of the formin Diaphanous (Dia) and Myosin-II. Their activity is controlled by the guanine exchange factor RhoGEF2, which is planar polarized and absent in non-intercalating regions. Finally, we provide evidence that Dia and Myo-II control the initiation of E-cadherin endocytosis by regulating the lateral clustering of E-cadherin.