Sir Bob Neill has said he wants to spend more time with his wife, who suffered a stroke in 2019.
One MP says promises made by the private jail are "worth less than the paper they are written on".
Justice Committee say the government and prison bosses must answer if inspectors' suggestions are ignored.
