Based on the report, action will be taken against the hospital, Delhi Health Minister Satyendar Jain said.
The baby, found to be alive by the family, died after a week of treatment today in a nursing home in Pitampura. A preliminary report by the Delhi government's panel found the hospital to be guilty of not following the norms while dealing with newborns.
"The preliminary report has found them guilty in certain aspects and the final report is expected in next two days. Once we get that we will take strictest possible action according to law," Mr Jain told reporters in Delhi.
The father alleged that no one from the Delhi government came forward to help.
Responding to the allegations, Mr Jain said, "There are certain legalities involved but we responded immediately and took action."
He had earlier said that the hospital's licence could be cancelled if it was found guilty of medical negligence.
The twins were born on November 30. The parents alleged that the hospital told them the babies, a boy and a girl, were stillborn and handed them the bodies in a plastic bag.
While the twins were being taken for burial, they found the boy to be breathing inside one of the packages, police said.
The health minister said that on November 22, a notice was sent to the hospital for allegedly violating norms related to services to be given to EWS or economically weaker section patients. "These two cases have been clubbed and we will take action accordingly," he said.
(With inputs from PTI)