In this study, we investigated on a systems level how complex protein interactions underlying cell polarity in yeast determine the dynamic association of proteins with the polar cortical domain (PCD) where they localize and perform morphogenetic functions. We constructed a network of physical interactions among >100 proteins localized to the PCD. This network was further divided into five robust modules correlating with distinct subprocesses associated with cell polarity. Based on this reconstructed network, we proposed a simple model that approximates a PCD protein's molecular residence time as the sum of the characteristic time constants of the functional modules with which it interacts, weighted by the number of edges forming these interactions. Regression analyses showed excellent fitting of the model with experimentally measured residence times for a large subset of the PCD proteins. The model is able to predict residence times using small training sets. Our analysis also revealed a scaffold protein that imposes a local constraint of dynamics for certain interacting proteins.