A nine-minute video on the CD allegedly shows Sandeep Kumar, 34, in what Aam Aadmi Party leaders described as "a compromising position with two women". It also had 11 photographs.
They had been leaked earlier in the evening to some TV channels, which sent a copy to the chief minister for reactions.
AAP went into a huddle. A meeting was on at the Chief Minister's Flagstaff Road bungalow till late in the night to decide on further action and also who would replace Mr Kumar as minister for Women and Child Development, Language and SC & ST.
Mr Kejriwal tweeted:
Recd "objectionable" CD of minister Sandeep Kr. AAP stands for propriety in public life. That can't be compromised(1/2)Arvind Kejriwal (@ArvindKejriwal) August 31, 2016
Soon after, his deputy Manish Sisodia told reporters, "The Delhi CM found an objectionable CD involving Sandeep Kumar. Within half an hour, action was taken. The minister was removed."
Removing him from Cabinet wid immediate effect(2/2)Arvind Kejriwal (@ArvindKejriwal) August 31, 2016
"AAP is a party of ideals. It has a zero tolerance policy towards corruption, scandals... Ministers found taking bribes were removed immediately as soon as it was found. Action will be taken swiftly against any member irrespective of status or position," he said.
What AAP said was its prompt action, the opposition called "damage control".
Former AAP members Yogendra Yadav and Prashant Bhushan said when they were in the party, they had warned Mr Kejriwal about the conduct of several candidates he had selected to contest the Delhi elections that his AAP swept last year.
"Chief Minister Kejriwal must resign for burdening Delhi with the most morally corrupt cabinet. Three out of six ministers removed in last one year and dozens of MLAs booked," said BJP leader Vijendra Gupta.
"I touch my wife's feet every morning," Sandeep Kumar had said after taking oath as cabinet minister. He was unavailable for comment today.