In the phenomenon known as cross-modal plasticity, the loss of one sensory system is followed by improved functioning of other intact sensory systems. MRI and functional MRI studies suggested a role of the prefrontal cortex and the temporal lobe in cross-modal plasticity. We used a mouse model to examine the effects of sensory deprivation achieved by whisker trimming and visual deprivation achieved by dark rearing in neonatal mice on the appearance of parvalbumin (PV) neurons and the formation of glutamic acid decarboxylase 67 (GAD67)-positive puncta around pyramidal neurons in the prefrontal cortex and hippocampus. Whisker trimming, but not dark rearing, decreased the density of PV neurons in the hippocampus at postnatal day 28 (P28). In the prefrontal cortex, whisker trimming and dark rearing decreased the density of PV neurons in layer 5/6 (L5/6) at P28 and in L2/3 at P56, respectively, whereas dark rearing increased the density of PV neurons in L5/6 at P56. Whisker trimming decreased the density of GAD67-positive puncta in CA1 of the hippocampus at both P28 and P56 and in L5/6 of the prefrontal cortex at P28. Dark rearing decreased the density of GAD67-positive puncta in CA1 of the hippocampus and in both L2/3 and L5/6 of the prefrontal cortex at P28, and in L2/3 of the prefrontal cortex at P56. These results demonstrate that somatosensory or visual deprivation causes changes in the PV-interneuronal network in the mouse prefrontal cortex and hippocampus. The results also suggest that the alteration of the PV-interneuronal network, especially in the prefrontal cortex, may contribute to cross-modal plasticity.