Pyruvate dehydrogenase kinases (PDHKs) are fascinating drug targets for numerous diseases, including diabetes and cancers. In this report, we describe the result of our structure-based drug design from tricyclic lead compounds that led to the discovery of highly potent PDHK2 and PDHK4 dual inhibitors in enzymatic assay. The C3-position of the tricyclic core was explored, and the PDHK2 X-ray structure with a representative compound revealed a novel ATP lid conformation in which the phenyl ring of Phe326 mediated the interaction of the Arg258 sidechain and the compound. Compounds with amide linkers were designed to release the ATP lid by forming an intramolecular pi-pi interaction, and these compounds showed single-digit nM IC50 values in an enzymatic assay. We also explored the C4-position of the tricyclic core to reproduce the interaction observed with the C3-position substitution, and the pyrrolidine compound showed the same level of IC50 values. By optimizing an interaction with the Asn255 sidechain through a docking simulation, compounds with 2-carboxy pyrrole moiety also showed single-digit nM IC50 values without having a cation-pi interaction with the Arg258 sidechain.