Endomembrane organization is important for various aspects of cell physiology, including membrane protein trafficking. To explore the molecular mechanisms regulating the trafficking of plasma membrane-localized proteins in plants, we screened for Arabidopsis mutants with defective localization of green fluorescent protein (GFP)-nodulin 26-like intrinsic protein (NIP)5;1. Fluorescence imaging-based screening led to the isolation of a mutant which accumulated abnormal intracellular aggregates labeled by GFP-NIP5;1. The aggregates appeared in epidermal cells in the root elongation zone and included the trans-Golgi network/early endosomes. Rough mapping and whole-genome sequencing identified the mutant as an allele of UDP-glucose 4-epimerase 4 (uge4)/root hair defective 1 (rhd1) /root epidermal bulgar 1 (reb 1), which was originally defined as a cell wall mutant. The responsible gene encodes UDP-glucose 4-epimerase 4 (UGE4), which functions in the biosynthesis of d-galactose, especially for the synthesis of the cell wall polysaccharide xyloglucan and arabinogalactan proteins (AGPs). The endomembrane aggregates in the mutants were absent in the presence of d-galactose, indicative of a requirement for a d-galactose-containing component in endomembrane organization. Genetic and pharmacological analyses suggested that the aggregates were not caused by the disruption of cell wall polysaccharides or the cytoskeleton. Overall, our results suggest that UGE4 activity in d-galactose synthesis is required for the structure of cell wall polysaccharides and endomembrane organization.