The Rapid Action Force was called in as Mr Kejriwal and his ministers sat on protest near the Rail Bhawan, close to the office of Prime Minister Manmohan Singh and the President's House.
Mr Kejriwal wants the union Home Ministry to suspend three police officers who refused to make a series of arrests ordered by his ministers.
Mr Kejriwal, 45, said he will not wait for the results of an enquiry, ordered by the Home Ministry, to determine if the police officers were following the law, as alleged by the police chief, or whether they abdicated responsibility. (Chief Minister turns chief protestor in unprecedented face-off: 10 developments)
"I want everyone to come and join us for this protest," said Mr Kejriwal, over-riding a tweet last night in which he urged the public not to join him in his demonstration because it would interfere with arrangements being made for Republic Day celebrations on Sunday. (Watch)
His impromptu speech was delivered after he was stopped by the police from heading to the Home Ministry where he planned a sit-in protest against its chief, Sushil Kumar Shinde. ('Have come prepared for a 10-day dharna': Law Minister Somnath Bharti to NDTV)
"They say I am an anarchist. Yes, I am. Today, I will create anarchy for Mr Shinde," he said .
He has been requested by the police to move his demonstration to Jantar Mantar, the designated spot for public protests. (Common gripes greet common man party: foreign media on Kejriwal)
On Wednesday night, Delhi Law Minister, Somnath Bharti conducted a self-described midnight "raid" in a South Delhi neighbourhood, and asked the police to arrest Ugandan citizens and raid a house where he alleged they were trafficking drugs and sex. The police refused, saying they had no warrant.
Four women have since filed a police case alleging that they were beaten up by a mob, and forced to submit urine samples.They tested negative for drugs.
In a separate case, another minister Rakhi Birla has accused the police of ignoring her orders to punish a family responsible for a dowry-related death