The hereditary hearing–vision loss disease Usher syndrome (USH) is caused by defects in several proteins, most of which form an integrated protein network called Usher interactome. Harmonin/Ush1C is a master scaffold in the assembly of the Usher protein complexes, because harmonin is known to bind to every protein in the Usher interactome. However, the biochemical and structural mechanism governing the Usher protein complex formation is largely unclear. Here, we report that the highly-conserved N-terminal fragment of harmonin (N-domain) immediately preceding its PDZ1 adopts an autonomously-folded domain. We discovered that the N-domain specifically binds to a short internal peptide fragment of the cadherin 23 cytoplasmic domain. The structures of the harmonin N-domain alone and in complex with the cadherin 23 internal peptide fragment uncovered the detailed binding mechanism of this interaction between harmonin and cadherin 23. We further elucidated the harmonin PDZ domain-mediated cadherin 23 binding by solving the structure of the second harmonin PDZ domain in complex with the cadherin 23 carboxyl tail. The multidentate binding mode between harmonin and cadherin 23 provides a structural and biochemical basis for the harmonin-mediated assembly of stable tip link complex in the auditory hair cells.