Caveolae are relatively stable membrane invaginations that compartmentalize signaling, regulate lipid metabolism and mediate viral entry. Caveolae are closely associated with actin fibers and internalize in response to diverse stimuli. Loss of cell adhesion is known to induce rapid and robust caveolae internalization and trafficking toward a Rab11-positive recycling endosome; however, pathways governing this process are poorly understood. Here, we report that filamin A is required to maintain the F-actin-dependent linear distribution of caveolin-1. High spatiotemporal resolution particle tracking of caveolin-1–GFP vesicles by total internal reflection fluorescence (TIRF) microscopy revealed that FLNa is required for the F-actin-dependent arrest of caveolin-1 vesicles in a confined area and their stable anchorage to the plasma membrane. The linear distribution and anchorage of caveolin-1 vesicles are both required for proper caveolin-1 inwards trafficking. De-adhesion-triggered caveolae inward trafficking towards a recycling endosome is impaired in FLNa-depleted HeLa and FLNa-deficient M2-melanoma cells. Inwards trafficking of caveolin-1 requires both the ability of FLNa to bind actin and cycling PKCα-dependent phosphorylation of FLNa on Ser2152 after cell detachment.