The phrase, used by Mr Gray in his letter of resignation, is thought to allude to the loss of computer discs containing confidential details of 15 million child benefit recipients.
Mr Gray, a former private secretary for economic affairs to the prime minister, took over from Sir David Varney as the Revenue's chairman in September 2006.
He said: 'This is not the way I would have planned to organise my departure from HMRC. I had hoped to be around for a while longer, and to have had the continuing privilege of leading HMRC towards the vision we have been developing.
'But I am extremely proud of what all of you in the organisation have achieved during my time as deputy chairman and chairman.
'Our record - for example in achieving sustained increases in tax receipts, in steadily improving the operations of the tax credit system, and in playing our full part in protecting the border - is a good one.
'At the same time we have made important steps in restructuring the department to face our future challenges, delivering more with reduced resources.
'I know we still face some major issues where we need to do better. But I am confident the forthcoming capability review report will highlight some major strengths in HMRC, while giving us helpful steers on how we can improve further.