Stable Diffusion works by first starting with white noise, and then using multiple rounds of probability optimization to eventually converge on a “most” correct given the initial noise image.
So to fix this, you mostly likely just need to run something through more rounds of optimization