Yes,I would imagine that Neville chased the interfering busybodies as their presence wouldn't have gone down too well,him being a magistrate and all that.
On the other hand,because the Bambers weren't aware of EVERYTHING because of Sheila living in London,they'd also not have been aware of the odd outburst either,although I would have thought that latterly " Freddie " would have put them in the picture as he'd visited WHF on a couple of occasions.
I liken Sheila to the late Princess Diana at times as both had stifled lives,controlled and even she threw hissy fits but it didn't mean that she was either mentally ill or unbalanced,and her life came to a sticky end with someone who NOBODY approved of.