WHO Director-General Tedros Adhanom Ghebreyesus named Mugabe to the largely ceremonial post at a meeting on Wednesday in Uruguay on chronic diseases attended by both men.
At the time, Tedros praised Zimbabwe as "a country that places universal health coverage and health promotion at the centre of its policies to provide healthcare to all".
But Tedros said in a statement that he had listened to those expressing concerns and heard the "different issues" raised.
"Over the last few days, I have reflected on my appointment of H.E. President Robert Mugabe as WHO Goodwill Ambassador for NCDs (non-communicable diseases) in Africa. As a result I have decided to rescind the appointment," Tedros said.
The decision had been taken after consultation with the Harare government and was "in the best interests of the World Health Organization", he said.
Jeremy Farrar, a global health specialist and director of the Wellcome Trust charity and the NCD alliance, representing health groups combating chronic diseases, welcomed the reversal.
"Dr Tedros deserves all our support to ensure he and WHO build a global health movement that is inclusive and works to improve health for everyone," Farrar said in a statement.
Zimbabwean Foreign Minister Walter Muzembi accepted the move while insisting that WHO had "benefited tremendously" from having nominated Mugabe and the media buzz it brought to health issues.
"So on the balance, it is wiser to let go, and help WHO focus on its mandate while we focus Zimbabwe on its membership obligations," Muzembi said in a government statement.
Mugabe was head of the African Union when the bloc endorsed Tedros - a former health and foreign minister of Ethiopia - over other African candidates for the top post, without any real regional contest, they said.
Mugabe, 93, is blamed in the West for destroying Zimbabwe's economy and for numerous human rights abuses during his 37 years leading the country as either president or prime minister.
Britain had said Mugabe's appointment as a goodwill ambassador was "surprising and disappointing" and that it risked overshadowing the WHO's global work. The United States, which has imposed sanctions on Mugabe for alleged human rights violations, also voiced disappointment.
"He (Tedros) has to remember where his funding comes from," said one health official who declined to be identified.
The Trump administration, which is already questioning financial support for some United Nations agencies, is WHO's largest single donor.
WHO is struggling to recover a reputation tarnished by its slowness in tackling the Ebola epidemic that killed more than 11,000 people in West Africa from 2014-2015 under Tedros' predecessor, Margaret Chan.
The agency is now grappling with a massive cholera outbreak in Yemen that has infected some 800,000 people in the past year and a plague outbreak in Madagascar that has killed nearly 100 in two months.
(Additional reporting by MacDonald Dzirutwe in Harare, editing by Larry King)