Pakistan has long been suspected by the West of playing a double game, fighting some militants while supporting those its generals have regarded as strategic assets to be used against rivals and neighbours, India and Afghanistan.
Kerry, on a visit to Pakistan, said all extremists groups should be targetted equally.
"Terror groups like the Pakistani and Afghan Taliban, the Haqqani network, Lashkar-e Taiba, and other groups, continue to pose a threat to Pakistan, to its neighbours, and to the United States, and we all of us have a responsibility to ensure that these groups do not gain a foothold but rather are pushed back into the recesses of (Pakistan's) memory," Kerry said in Islamabad.
"This task is obviously far from finished," he added.
Speaking after visiting India over the weekend, Kerry added that he was deeply concerned by recent violence on the border, and called on India and Pakistan to find a diplomatic solution.
"We continue to be deeply concerned by the recent spate of increased violence along the working boundary and the line of control," Kerry said, referring to a ceasefire line that has become the de facto border between the two countries.
"It is profoundly in the interests of Pakistan and India to move this relationship forward," he said.