The low molecular weight protein tyrosine phosphatase (LMW-PTP) is a regulator of a number of signaling pathways and has been implicated as a potential target for oncology and diabetes/obesity. There is significant therapeutic interest in developing potent and selective inhibitors to control LMW-PTP activity. We report the discovery of a novel class of LMW-PTP inhibitors derived from sulfophenyl acetic amide (SPAA), some of which exhibit greater than 50-fold preference for LMW-PTP over a large panel of PTPs. X-ray crystallography reveals that binding of SPAA-based inhibitors induces a striking conformational change in the LMW-PTP active site, leading to the formation of a previously undisclosed hydrophobic pocket to accommodate the α-phenyl ring in the ligand. This induced-fit mechanism is likely a major contributor responsible for the exquisite inhibitor selectivity.