I can fully understand why Jeremy wants WHF left as it originally was,,,as far as he's concerned,time had stood still when he was imprisoned and he wanted to remember how the place was should he ever return.
He's obviously the sentimental type.
It's memories for him rather than anything sinister behind his wishes. He's obviously someone who doesn't like change and would rather things stayed as they were when it was his home.
Being a Georgian-built property,,any alteration either inside or out would not add to the ambience and beauty of a Grade 2 listed building,,as usually the contents,furnishings,etc reflect the type of house it is.