The move comes at a time when the AAP government and bureaucrats are at loggerheads since the alleged attack on Chief Secretary Anshu Prakash by some ruling party MLAs at the chief minister's residence in February.
"Exemplary action will be taken against the guilty in each case of corruption or irregularity pointed by CAG. No one will be spared (sic)," Mr Kejriwal tweeted.
Sources said that at least 50 cases from the CAG report have been identified by the government so far and the report is being studied for more such cases to be sent for CBI probe.
"At least 50 cases from the CAG report have been identified to be sent to CBI for inquiry. The report is being studied and the number of cases to be sent to CBI may increase. The chief minister himself is looking into the report," they said.
In his address to the House on the CAG report, Mr Sisodia cited several "shortcomings and lapses" including construction of roads and public distribution system, and hit out at Lieutenant Governor Anil Baijal and bureaucrats.
"The CAG report has helped us in identifying rigidities in the system. We will remove the shortcomings and take action against those responsible, be it ministers or officers," Mr Sisodia said.
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.