Mr Yadav, who is the elder son of RJD chief Lalu Prasad, was allotted the bungalow at 3 Deshratna Marg when he was health minister.
But after the RJD-JD(U)-Congress coalition collapsed last year and Nitish Kumar later formed the government with the BJP, Mr Yadav was among the politicians who were told by the government to vacate their government bungalows so that others could move in.
In October last year, some former RJD ministers moved the Patna high court against the state government's order that asked them to move out. The court stayed the government's order, after which the government had asked the occupants to pay rent at market rates. Those who have stayed put are yet to comply with the order.
The chief minister's party has dismissed Mr Yadav's ghost remarks as a publicity seeking gimmick. "Tej Pratap Yadav suffers from acute attention deficiency and is trying to compete with his younger brother Tejaswi who is hogging the limelight in the media," the JD(U) said.
Although Mr Yadav was allocated the bungalow, he used it mostly for meeting his supporters during the day and used to stay at the bungalow allocated to his mother and former chief minister Rabri Devi.
Earlier, Mr Yadav had made some alterations to the government bungalow to make it 'Vaastu' compliant. He had shut the main entrance and used the rear exit.