"Do you think I am in politics for a post? No, I am not. I am here for the welfare of the public and that cannot be compromised," the minister fumed at his home on Sunday, declaring a dharna or sit-in. Mr Rajbhar heads the Suheldev Bharatiya Samaj Party, which partners the BJP in the Yogi Adityanath government and has a strong presence in eastern UP.
Mr Khatri, an Indian Administrative Service (IAS) officer, is the district magistrate of Ghazipur, Mr Rajbhar's home turf. He reportedly ordered action against a group of men believed to be close to the minister for misbehaving with officials.
Mr Rajbhar denies that is the reason; he says he is fed up of "administrative incompetence and rampant corruption" on the district magistrate's watch.
Sources in the Uttar Pradesh government say they are speaking to the minister and hope to resolve the problem. The district magistrate has not commented so far.
This is the latest in a series of run-ins in UP between the political leadership and officials. On Saturday, a police officer who stood up to a group of BJP workers and sent five of them to jail was among 234 officers transferred by the state government. Shreshtha Thakur had last week acted firmly against angry BJP workers who were arguing and protesting after their colleague was fined for not carrying proper documents while riding a motorbike and then arrested for misbehaving with policemen. Five were arrested.