Many citizens have not known a time without Mugabe, who has dominated every aspect of public life since coming to power in 1980 on the country's independence from Britain.
The nation was left stunned after the veteran president was confined to his residence late Tuesday, soldiers took up positions at strategic points across Harare and senior officers commandeered state television.
The Southern African Development Community, a bloc currently chaired by Zimbabwe's neighbour South Africa, was to meet in Botswana later Thursday to discuss the dramatic situation.
Nothing has been heard from Mugabe or his 52-year-old wife Grace since the start of the army operation.
But many Zimbabweans hoped the crisis would pave the way to a more prosperous future.
"Our economic situation has deteriorated every day -- no employment, no jobs," Tafadzwa Masango, a 35-year-old unemployed man, told AFP.
"We hope for a better Zimbabwe after the Mugabe era. We feel very happy. It is now his time to go."
However, a spokesman for Mugabe's ruling party, The Zimbabwe African National Union - Patriotic Front or ZANU-PF, Simon Khaya Moyo, insisted it was business as usual.
"It's normal, everything is normal with the party," he told AFP.
Harare's residents have largely ignored the military presence on the streets and continued commuting, socialising and working, while analysts speculated that Mugabe and the army could be negotiating a transition.
'A Greedy Lady'
Shadrack Gutto, director of the Centre for African Renaissance Studies at the University of South Africa, said that Grace had undermined the president's power through her own ambitions.
"I think that the current crisis has been triggered by the wife of President Mugabe... she wanted to grab power," he said. "She is a greedy lady."
Attention is now shifting to prominent leaders like former vice presidents Emmerson Mnangagwa and Joice Mujuru who could play a role in any transitional government.
Mujuru, a struggle-era leader who fought as a guerrilla fighter while still a teenager, told journalists on Thursday that Zimbabwe was "in need of transitional arrangements" - but did not give any details.
On Wednesday the African Union issued an unusually terse statement that said the situation on the ground "seems like a coup" and called on the military to pull back and respect the constitution.
In Paris, the head of the African Union, President Alpha Condy of Guinea, warned Thursday the body "will never accept the military coup d'etat" in Zimbabwe and called for a return to the "constitutional order."
"We know there are internal problems. They need to be resolved politically by the ZANU-PF party and not with an intervention by the army," added Conde, reiterating opposition that he voiced a day earlier.
Britain, Zimbabwe's former colonial ruler, called for elections scheduled for 2018 to go ahead.
"What matters more than anything else is for the presidential elections to go ahead," British Foreign Secretary Boris Johnson wrote in a column in the Daily Telegraph on Thursday.
Zimbabwe's army was set on a collision course with Mugabe last week when he abruptly fired Mnangagwa, a linchpin of the defence and security establishment.
Mnangagwa, 75, was previously one of Mugabe's most loyal lieutenants, having worked alongside him for decades.
But he fled to South Africa following his dismissal and published a scathing five-page rebuke of Mugabe's leadership and Grace's political ambition.
The state-owned Herald newspaper remained loyal to Mugabe on Thursday but also backed the military intervention.
"The military does not readily interfere in civilian political affairs. Yet this week they had to break with this long tradition," it said in an editorial, adding that ZANU-PF "was being soiled by those who should be helping the President".
Army chief General Constantino Chiwenga had given an unprecedented press conference on Monday, flanked by dozens of officers, warning Mugabe that he would intervene if the president continued to purge the ruling ZANU-PF party.
Mnangagwa has been embroiled in a long-running feud with Grace Mugabe.
Both were seen as leading contenders to replace Mugabe but Mnangagwa had the tacit support of the armed forces, which viewed Grace, a political novice, with derision.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.