Mechanistic target of rapamycin complex 1 (mTORC1) integrates diverse environmental signals to control cellular growth and organismal homeostasis. In response to nutrients, Rag GTPases recruit mTORC1 to the lysosome to be activated, but how Rags are regulated remains incompletely understood. Here, we show that Sestrins bind to the heterodimeric RagA/B-RagC/D GTPases, and function as guanine nucleotide dissociation inhibitors (GDIs) for RagA/B. Sestrin overexpression inhibits amino-acid-induced Rag guanine nucleotide exchange and mTORC1 translocation to the lysosome. Mutation of the conserved GDI motif creates a dominant-negative form of Sestrin that renders mTORC1 activation insensitive to amino acid deprivation, whereas a cell-permeable peptide containing the GDI motif inhibits mTORC1 signaling. Mice deficient in all Sestrins exhibit reduced postnatal survival associated with defective mTORC1 inactivation in multiple organs during neonatal fasting. These findings reveal a nonredundant mechanism by which the Sestrin family of GDIs regulates the nutrient-sensing Rag GTPases to control mTORC1 signaling.