The formation of mature secretory granules is essential for proper storage and regulated release of hormones and neuropeptides. In pancreatic β cells, cholesterol accumulation causes defects in insulin secretion and may participate in the pathogenesis of type 2 diabetes. Using a novel cholesterol analog, we show for the first time that insulin granules are the major sites of intracellular cholesterol accumulation in live β cells. This is distinct from other, non‐secretory cell types, in which cholesterol is concentrated in the recycling endosomes and the trans‐Golgi network. Excess cholesterol was delivered specifically to insulin granules, which caused granule enlargement and retention of syntaxin 6 and VAMP4 in granule membranes, with concurrent depletion of these proteins from the trans‐Golgi network. Clathrin also accumulated in the granules of cholesterol‐overloaded cells, consistent with a possible defect in the last stage of granule maturation, during which clathrin‐coated vesicles bud from the immature granules. Excess cholesterol also reduced the docking and fusion of insulin granules at the plasma membrane. Together, the data support a model in which cholesterol accumulation in insulin secretory granules impairs the ability of these vesicles to respond to stimuli, and thus reduces insulin secretion.