In this paper, we describe the primitive ideal space of the $C^*$-algebra $C^*(mathcal G)$ associated to the ultragraph $mathcal{G}$. We investigate the structure of the closed ideals of the quotient ultragraph $ C^* $-algebra $C^*left(mathcal G/(H,S)right)$ which contain no nonzero set projections and then we characterize all non gauge-invariant primitive ideals. Our results generalize the ...