We analyze within a semiclassical approximation the form factor for the fluctuations of quantum matrix elements around their classical average. We find two contributions. One is proportional to the form factor for the density of states, with an amplitude determined by the squared average of the matrix elements. The other is constant and related to the fluctuations of finite time classical traje...