Mattis, on a one-day visit to Pakistan, said the South Asian nation had made progress in the fight against terrorism inside its borders but needed to make more.
More than 100 days since President Donald Trump announced a South Asia strategy that calls for a firmer line towards Islamabad, US officials and analysts say there has been only limited success and it is not clear how progress will be made.
US officials have long been frustrated by what they see as Pakistan's reluctance to act against terror groups such as the Afghan Taliban and the Haqqani network that they believe exploit safe haven on Pakistani soil to launch attacks in Afghanistan.
"The Secretary reiterated that Pakistan must redouble its efforts to confront militants and terrorists operating within the country," the Pentagon said in a statement.
Mattis, who visited Pakistan for the first time as defence secretary, said before the trip that the goal for his meetings with Pakistani officials would be to find "common ground".
In his discussion with Mattis, Pakistani Prime Minister Shahid Khaqan Abbasi said the two allies shared objectives.
"We're committed (to) the war against terror," he said. "Nobody wants peace in Afghanistan more than Pakistan."
Mattis also met with high-ranking officials from Pakistan's military, including army chief General Qamar Javed Bajwa and Lieutenant-General Naveed Mukhtar, the head of the Inter-Services Intelligence spy agency that US officials say has links with Haqqani and Taliban terrorists.
In August, Trump outlined a new strategy for the war in Afghanistan, chastising Pakistan over its alleged support for Afghan terrorists. But beyond that, the Trump administration has done little to articulate its strategy, experts say.
"We have been very direct and very clear with the Pakistanis... we have not seen those changes implemented yet," General John Nicholson, the top US general in Afghanistan, said this week.
Pakistani officials have pushed back on the US accusations and say they have done a great deal to help the United States in tracking down terrorists.
US official expressed hope relations could improve after a US-Canadian couple kidnapped in Afghanistan were freed in Pakistan in October with their three children.
However, while the Trump administration has used tougher words with Pakistan, it is has yet to change Islamabad's calculus. Some experts say the United States loses clout in Pakistan when it is seen as bullying.
While Mattis travelled to the region earlier this year, he did not stop in Pakistan, but visited India, a relationship that has grown under the Trump administration.
"There is not an effective stick anymore because Pakistan doesn't really care about U.S aid, it has been dwindling anyway and it is getting the money it needs elsewhere ... treat it with respect and actually reward it when it does do something good," said Madiha Afzal, with the Brookings Institution.
Mattis's brief visit to Islamabad comes a week after a hardline Pakistani religious group called off nationwide protests after the government met its demand that a minister accused of blasphemy resign.