Many proteins are retrieved to the trans-Golgi Network (TGN) from the endosomal system through several retrograde transport pathways to maintain the composition and function of the TGN. However, the molecular mechanisms involved in these distinct retrograde pathways remain to be fully understood. Here we have used fluorescence and electron microscopy as well as various functional transport assays to show that Rab11a/b and its binding protein FIP1/RCP are both required for the retrograde delivery of TGN38 and Shiga toxin from early/recycling endosomes to the TGN, but not for the retrieval of mannose-6-phosphate receptor from late endosomes. Furthermore, by proteomic analysis we identified Golgin-97 as a FIP1/RCP-binding protein. The FIP1/RCP-binding domain maps to the C-terminus of Golgin-97, adjacent to its GRIP domain. Binding of FIP1/RCP to Golgin-97 does not affect Golgin-97 recruitment to the TGN, but appears to regulate the targeting of retrograde transport vesicles to the TGN. Thus, we propose that FIP1/RCP binding to Golgin-97 is required for tethering and fusion of recycling endosome-derived retrograde transport vesicles to the TGN.