Why would do that.
Everyone went to bed between 8pm - 11pm. As highlighted by people's attire or being shot in bed. Or both.
It has been known for people to go to bed during that time period.
I've just explained why that could have happened in the post you have just quoted.
Additionally Nevill did not go to bed between 8 and 11 at that time of year. It's unlikely he would be done outside until at least about 10.45 p.m., maybe 11 p.m. I repeat that this is the south of England during the summer harvest period. He was in the habit of sitting up in the den, lounge (sitting room), or upstairs office with a brandy or whisky and he also walked Bruce.
I could imagine him having a habit of sleeping downstairs and drawing his pyjamas from somewhere like the airing cupboard in the downstairs shower room. He could change into these after a shower, then have a drink and relax, then fall asleep where he was. Maybe sometimes he just fell asleep in his work clothes, though we know that on this night, he didn't, he changed.
Equally, I agree, he could have gone upstairs, but this introduces a number of problems into the crime scene if Jeremy did it.
I think I have gone over this with you enough now.