While addressing an emergency Cabinet meeting here, Sharif, 67, termed the Joint Investigation Team (JIT) report a pack of "allegations and speculation", Dawn Online reported.
Pointing to the opposition parties who have been demanding his resignation following the release of the report, Sharif said: "The people of Pakistan have elected me and only they can remove me from this post."
Nawaz Sharif claimed that his family "earned nothing after entering politics, but lost a lot".
The language used in the JIT report displays malafide intentions, he said.
"Those demanding my resignation on false and unwarranted claims should first look at themselves," the Prime Minister said and announced that that he would not resign on demands of conspirers.
According to the paper, the Cabinet members suggested that Sharif must fight the legal battle to vindicate himself in the Panama Papers case.
The six-member JIT that probed the Sharif family's business dealings in its 10-volume report submitted to the apex court on July 10 recommended that a corruption case should be filed against Nawaz Sharif and his sons Hassan Nawaz and Hussain Nawaz, as well as daughter Maryam Nawaz, under the National Accountability Bureau (NAB) ordinance 1999.
The decision to convene today's meeting was taken during an "informal meeting" at the prime minister's house.
All major opposition political parties have asked him to step down and stay away from power until his name was cleared.