Polyglycylation is a posttranslational modification that generates glycine side chains on proteins. Here we identify a family of evolutionarily conserved glycine ligases that modify tubulin using different enzymatic mechanisms. In mammals, two distinct enzyme types catalyze the initiation and elongation steps of polyglycylation, whereas Drosophila glycylases are bifunctional. We further show that the human elongating glycylase has lost enzymatic activity due to two amino acid changes, suggesting that the functions of protein glycylation could be sufficiently fulfilled by monoglycylation. Depletion of a glycylase in Drosophila using RNA interference results in adult flies with strongly decreased total glycylation levels and male sterility associated with defects in sperm individualization and axonemal maintenance. A more severe RNAi depletion is lethal at early developmental stages, indicating that protein glycylation is essential. Together with the observation that multiple proteins are glycylated, our functional data point towards a general role of glycylation in protein functions.