I can think of only one reason why Jeremy didn't get rid of the silencer. He might have thought that they could tell the bullets had passed through a silencer. However, there's absolutely no reason he would have put it back in the cupboard because he'd have to explain why Sheila would have put it there before she shot herself, and he certainly wouldn't put it back in there without cleaning it.