Gerodermia osteodysplastica is a hereditary segmental progeroid disorder affecting skin, connective tissues, and bone that is caused by loss-of-function mutations in GORAB. The golgin, RAB6-interacting (GORAB) protein localizes to the Golgi apparatus and interacts with the small GTPase RAB6. In this study, we used different approaches to shed more light on the recruitment of GORAB to this compartment. We show that GORAB best colocalizes with trans-Golgi markers and is rapidly displaced upon Brefeldin A exposition, indicating a loose association with Golgi membranes. A yeast two-hybrid screening revealed a specific interaction with the small GTPase ADP-ribosylation factor (ARF5) in its active, GTP-bound form. ARF5 and RAB6 bind to GORAB via the same internal Golgi-targeting RAB6 and ARF5 binding (IGRAB) domain. Two GORAB missense mutations identified in gerodermia osteodysplastica patients fall within this IGRAB domain. GORAB carrying the mutation p.Ala220Pro had a cytoplasmic distribution and failed to interact with both RAB6 and ARF5. In contrast, the p.Ser175Phe mutation displaced GORAB from the Golgi compartment to vesicular structures and selectively impaired ARF5 binding. Our findings indicate that the IGRAB domain is crucial for the Golgi localization of GORAB and that loss of this localization impairs its physiological function.