Mugabe's grip on power was broken last week when the military took over, angered at his wife Grace's emergence as the leading candidate to succeed the 93-year-old president.
On Saturday, tens of thousands of overjoyed demonstrators flooded the streets of Zimbabwe in peaceful celebrations marking the apparent end of his long and authoritarian rule.
Outside a ZANU-PF meeting in Harare, a delegate told AFP that Mugabe had been ousted as party chief and replaced by Emmerson Mnangagwa, who was previously Grace Mugabe's chief rival to succeed the ageing president.
"A resolution has been adopted to recall the president and elevate Mnangagwa as the party president," said the delegate, who declined to be named.
Mugabe, the world's oldest head of state, remains national president for the time being but now faces overwhelming opposition from the generals, much of the Zimbabwean public and from his own party.
"(Mugabe's) wife and close associates have taken advantage of his frail condition to usurp power and loot state resources," party official Obert Mpofu told the ZANU-PF meeting.
Army chiefs who led the takeover were due to hold further talks with the president later Sunday.
The two sides first met on Thursday, smiling in photographs that attempted to present a dignified image of the tense process of negotiating Mugabe's departure.
Veterans of the independence war, who were also formerly key Mugabe allies, added their voice in support of him resigning, demanding that he leave office on Sunday.
Zimbabweans have experienced a historic week in which the military seized power and put Mugabe under house arrest in response to his sacking of vice president Mnangagwa, who has close military ties.
On Saturday, in scenes of public euphoria not seen since Zimbabwe's independence in 1980, huge crowds marched and sang their way through Harare and other cities.
The demonstrations included citizens of all ages, jubilant that Mugabe appeared to be on his way out.
In central Harare, a group of young men tore down a green metal street sign bearing Robert Mugabe's name and smashed it repeatedly on the road.
Such open dissent, just a week ago, would have been routinely crushed by security forces.
Peace, Jobs, A Normal Life
"What you saw yesterday, it shows that the people have spoken," Mordecai Makore, 71, a retired teacher told AFP after attending a Sunday morning service at the Catholic cathedral in central Harare.
"All we want is peace, a good life with a working economy that creates jobs for our people. We will continue praying for that. I want my children and grandchildren to live a normal good life."
The majority of Zimbabweans have only known life under Mugabe's rule, which has been defined by violent suppression, economic collapse and international isolation.
Sources suggest Mugabe has been battling to delay to his exit and to secure a deal guaranteeing future protection for him and his family.
He attended a university graduation ceremony on Friday, in a show of defiance after the talks with General Constantino Chiwenga, who led the military power grab.
The factional succession race that triggered Zimbabwe's sudden crisis was between party hardliner Mnangagwa, known as the Crocodile, and a group called "Generation 40" or "G40" because its members are generally younger, which campaigned for Grace's cause.
"She is very acceptable. Very much accepted by the people," Mugabe said of Grace in a faltering interview to mark his 93rd birthday last February.
The president, who is feted in parts of Africa as the continent's last surviving liberation leader serving as a head of state, is in fragile health. But he previously said he would stand in elections next year that would see him remain in power until he was nearly 100-years-old.
He became prime minister on Zimbabwe's independence from Britain in 1980 and then president in 1987.
Zimbabwe's economic output has halved since 2000 when many white-owned farms were seized, leaving the key agricultural sector in ruins.