The Arc/Arg3.1 gene product is rapidly upregulated by strong synaptic activity and critically contributes to weakening synapses by promoting AMPA-R endocytosis. However, how activity-induced Arc is redistributed and determines the synapses to be weakened remains unclear. Here, we show targeting of Arc to inactive synapses via a high-affinity interaction with CaMKIIβ that is not bound to calmodulin. Synaptic Arc accumulates in inactive synapses that previously experienced strong activation and correlates with removal of surface GluA1 from individual synapses. A lack of CaMKIIβ either in vitro or in vivo resulted in loss of Arc upregulation in the silenced synapses. The discovery of Arc's role in "inverse" synaptic tagging that is specific for weaker synapses and prevents undesired enhancement of weak synapses in potentiated neurons reconciles essential roles of Arc both for the late phase of long-term plasticity and for reduction of surface AMPA-Rs in stimulated neurons.