There has been a "transcendent failure of governance", the Congress leader said, a day after a Panchkula court's guilty verdict against the Dera Sacha Sauda chief triggered widespread violence in which 31 people were killed.
The Haryana government's inability to maintain law and order in a situation of predictable chaos was to be blamed, Mr Tharoor said on the sidelines of the Mountain Echoes literary festival in Thimpu.
"The lack of management of predictable crowd violence, where anyone could see people gathering with weapons in advance of the verdict is the major reason behind what has happened," Mr Tharoor said.
"Ultimately it is the chief minister and his cabinet who are responsible for the law and order situation in the state.
It is a complete failure of the government and the CM should be sacked for it," he added.
According to Mr Tharoor, the mob was "clearly preparing" for violence.
"Why wasn't any action taken first under Section 144?
Why, for example, were all gatherings not prohibited? Why was there no preventive deployment of armed police on the streets?
All of these are such obvious questions to be asked.
"To my mind this has been a transcendent failure of governance."
The government of Haryana, he said, was principally responsible for the lack of adequate preventive measures.
"It has also been said that they had actually allowed people to assemble. If so, it is totally condemnable and I hope an investigation is done...," Mr Tharoor said.
The Congress had yesterday called for Mr Khattar's resignation, with party president Sonia Gandhi expressing grave concern over "widespread destruction of public property and senseless attacks".