Abadi's office said he was visiting "liberated" Mosul to congratulate his "heroic fighters", but the premier later indicated he would only declare victory once final pockets of resistance were cleared.
"Victory is certain, and the (IS) remnants are surrounded... and it is just a matter of time for us to announce the great victory to our people," Abadi said in a statement.
The delay "comes out of my respect and support for our... forces who are continuing the clearing operation," he said.
"There are only one or two pockets of Daesh remnants left," and "the major victory is in hand," the premier added, using an Arabic acronym for ISIS.
That victory comes at an enormous cost: much of Iraq's second city in ruins, thousands dead and wounded, and nearly a million people forced from their homes.
And enormous challenges lie ahead, not just in rebuilding Mosul but in tackling the continued presence elsewhere of IS, whose threat has been sharply reduced but not eliminated.
Photographs released by his office showed Abadi dressed in a black military uniform and cap, shaking hands with police and army officers.
His office said Abadi met commanders in Mosul and issued a series of orders on "sustaining victories and eliminating the defeated remnants" of IS, as well as "establishing security and stability in the liberated city."
'Victory for all Iraqis'
Iraqi forces waved flags and flashed victory signs after Abadi arrived in the city.
"This victory is for all Iraqis, not just for us," Mohanned Jassem, a member of the elite Counter-Terrorism Service, told AFP at the police base where Abadi met commanders.
Jassem, who fought in most of the other main battles of the war against IS, said Mosul was the toughest.
"I took part in fighting in Ramadi and Tikrit and Salaheddin and Baiji and Al-Qayyarah... but the fighting here in (ISIS's) stronghold was the most violent," he said, an Iraqi flag draped over his shoulders.
IS swept across much of Iraq's Sunni Arab heartland in a lightning offensive in mid-2014, proclaiming a "caliphate" straddling Iraq and neighbouring Syria.
Imposing its brutal interpretation of Islamic law, the group committed widespread atrocities and organised or inspired deadly attacks in Iraq, Syria and abroad.
A US-led coalition launched military operations against IS in Syria and Iraq in mid-2014, carrying out a campaign of air strikes against the jihadists and sending advisers to work with local ground forces.
French President Emmanuel Macron was among the first world leaders to offer his congratulations.
"Mosul liberated from Daesh," he tweeted. "Homage from France to all those, with our troops, who contributed to this victory."
British Defence Secretary Michael Fallon congratulated Abadi and the "Iraqi forces who have been fighting on the ground with great bravery".
The European Union called the victory "a decisive step in the campaign to eliminate terrorist control in parts of Iraq".
IS has lost most of the territory it once controlled, and the coalition is aiming to oust the jihadists from their Syrian stronghold Raqa, which is under assault by US-backed Arab and Kurdish forces.
Iraqi forces launched their campaign to recapture Mosul in October, seizing its eastern side in January and launching the battle for its western part the next month.
But the fight grew tougher when security forces entered the densely populated Old City on the western bank of the Tigris River, which divides the city.
In recent days, security forces have killed jihadists trying to escape their dwindling foothold in Mosul, as Iraqi units fought to retake the last IS-held territory near the Tigris.
Earlier Sunday, Iraq's Joint Operations Command said security forces had killed "30 terrorists" trying to escape across the river.
Even in the final days of the battle, thousands of civilians remained trapped inside the Old City and some of those who fled arrived grief-stricken after losing relatives in jihadist sniper fire and bombardments.
Not yet 'the death knell'
The United Nations said this week that 915,000 residents people had been displaced since the battle for Mosul began in October.
The recapture of Mosul will not mark the end of the threat posed by IS, which controls territory elsewhere in Iraq and is able to carry out frequent bombings in government-held areas.
In Iraq it holds towns including Tal Afar and Hawijah in the north, as well as territory in western Anbar province.
It also continues to hold significant territory in Syria, including in Raqa, where the US-backed Syrian Democratic Forces (SDF) are battling to oust the jihadist group after penetrating its fortified historic centre.
While the loss of Mosul is a major blow to the jihadists, it is not a fatal one.
"We should not view the recapture of Mosul as the death knell for ISIS," said Patrick Martin, Iraq analyst at the Institute for the Study of War.
"If security forces do not take steps to ensure that gains against ISIS are sustained for the long-term, then IS could theoretically resurge and recapture urban terrain," he said.