Glucan phosphatases are central to the regulation of starch and glycogen metabolism. Plants contain two known glucan phosphatases, Starch EXcess4 (SEX4) and Like Sex Four2 (LSF2), which dephosphorylate starch. Starch is water-insoluble and reversible phosphorylation solubilizes its outer surface allowing processive degradation. Vertebrates contain a single known glucan phosphatase, laforin, that dephosphorylates glycogen. In the absence of laforin, water-soluble glycogen becomes insoluble, leading to the neurodegenerative disorder Lafora Disease. Because of their essential role in starch and glycogen metabolism glucan phosphatases are of significant interest, yet a comparative analysis of their activities against diverse glucan substrates has not been established. We identify active site residues required for specific glucan dephosphorylation, defining a glucan phosphatase signature motif (CζAGΨGR) in the active site loop. We further explore the basis for phosphate position-specific activity of these enzymes and determine that their diverse phosphate position-specific activity is governed by the phosphatase domain. In addition, we find key differences in glucan phosphatase activity toward soluble and insoluble polyglucan substrates, resulting from the participation of ancillary glucan-binding domains. Together, these data provide fundamental insights into the specific activity of glucan phosphatases against diverse polyglucan substrates.