"Are you with us or with Qatar?" Saudi King Salman posed the question to Mr Sharif, who had visited the Gulf Kingdom for finding a diplomatic solution to the Qatar crisis, The Express Tribune reported citing diplomatic sources.
"Pakistan has told Saudi Arabia it will not take sides in the brewing diplomatic crisis in the Middle East after Riyadh asked Islamabad 'are you with us or with Qatar'," said the daily.
Pakistan has been treading a careful path since Saudi Arabia and other Gulf countries severed diplomatic ties with Qatar after accusing the oil-rich country of supporting terrorist groups. Saudi Arabia, it said, wanted Pakistan's support.
Mr Sharif, accompanied by Pakistan army chief General Qamar Javed Bajwa and other senior officials, travelled to Jeddah on Monday to discuss the emerging situation in the Gulf world. His mediation visit to Saudi did not achieve any immediate breakthrough.
Citing a senior government official, who was briefed on the talks at the monarch's palace in Jeddah, the daily said that Pakistan would not take sides in any event that would create divisions within the Middle East.
"Nevertheless, in order to placate Saudi Arabia, Pakistan offered to use its influence over Qatar to defuse the situation. For this purpose, the prime minister will undertake visits to Kuwait, Qatar and Turkey," it said.
The Saudi Press Agency, while announcing the severing of relations with Qatar, had said it had been done for "protection of national security from the dangers of terrorism and extremism".
Qatar, meanwhile, has indicated that it was willing to address concerns of the countries that ended diplomatic relations with it.
The current crisis in the Gulf is said to be the gravest that the Gulf Cooperation Council has faced in its nearly four decades of existence.
The UAE, Bahrain and Egypt also joined Saudi Arabia and cut off diplomatic ties and transport links with Qatar on June 5 on the pretext that its policies are fuelling extremism and terrorism.
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.