Speaking at the Asia Society seminar in New York on Tuesday, he said: "It is very easy to say that Pakistan is supporting Haqqanis, Hafiz Saeed and Lashkar-e-Taiba. They are liabilities...we have said they are liabilities... but give us time to get rid of these liabilities because we do not have the assets to match these liabilities and you are increasing them further."
Mr Asif, who is attending the UN General Assembly, added: "Don't blame us for Haqqanis or don't blame us for the Hafiz Saeeds. These were the people who were your darlings just 20 to 30 years back. They were being dined and wined in the White House and now you say go to hell Pakistanis because you are nurturing these people."
Hafiz Saeed, the mastermind of the 2008 Mumbai terror attack that left 166 dead, roams free in Pakistan and recently launched a political party, saying he would contest elections in 2018.
Under growing international pressure to crack down on terrorists operating from its soil, Pakistan earlier this week opposed the registration of Hafiz Saeed's political party Milli Muslim League. In a letter to its election body, Islamabad cited the refusal of security clearance by intelligence agencies and also said "some countries" have raised the subject diplomatically.
Hafiz Saeed has been declared a global terrorist by the UN and the US for his role in the Mumbai attack. He was placed under house arrest earlier this year and it was extended several times, but no formal charges have been filed against him so far. On Tuesday, Pakistani authorities extended his house arrest by another month.
Mr Asif's comments on Hafiz Saeed follow Indian Foreign Minister Sushma Swaraj's blistering attack on Pakistan at the UN General Assembly. Addressing leaders from 193 countries on Saturday, Ms Swaraj said: "We set up IITs, IIMs, AIIMS and ISRO. What did Pakistan make? They set up Lashkar-e-Taiba, Jaish-e-Mohammed, Haqqani Network, Hizbul Mujahideen."