Xanthine oxidase (XOD) inhibition has long been considered an effective anti-hyperuricemia strategy. To identify effective natural XOD inhibitors with little side effects, we performed a XOD inhibitory assay-coupled isolation of compounds from Smilacis Glabrae Rhizoma (SGR), a traditional Chinese medicine frequently prescribed as anti-hyperuricemia agent for centuries. Through the in vitro XOD inhibitory assay, we obtained a novel XOD inhibitor, 5-O-caffeoylshikimic acid (#1, 5OCSA) with IC50 of 13.96 μM, as well as two known XOD inhibitors, quercetin (#3) and astilbin (#6). Meanwhile, we performed in silico molecular docking and found 5OCSA could interact with the active sites of XOD (PDB ID: 3NVY) with a binding energy of −8.6 kcal/mol, suggesting 5OCSA inhibits XOD by binding with its active site. To evaluate the in vivo effects on XOD, we generated a hyperuricemia mice model by intraperitoneal injection of potassium oxonate (300 mg/kg) and oral gavage of hypoxanthine (500 mg/kg) for 7 days. 5OCSA could inhibit both hepatic and serum XOD in vivo, together with an improvement of histological and multiple serological parameters in kidney injury and HUA. Collectively, our results suggested that 5OCSA may be developed into a safe and effective XOD inhibitor based on in vitro, in silico and in vivo evidence.