The donkeys, bought in Tanzania, were given to residents of a village in Gitega province, as part of a project by a local NGO to help women and children transport agricultural products, water or wood.
However a presidential advisor described the project as "an insult to the nation".
Meanwhile Gabby Bugaga, spokesman for the Senate president, wrote on Twitter that the French were "taking us for donkeys".
"Be honest, is the donkey a symbol of a quality or a flaw," he wrote.
Donkeys are not indigenous to Burundi.
On Sunday, Agriculture Minister Deo Guide Rurema asked a local administrator to "facilitate the immediate withdrawal of all donkeys that have been distributed ... without respecting the technical procedure of the distribution of exotic animals."
After the controversy he said that to his knowledge "all procedures were respected".
A European diplomat, speaking on condition of anonymity, said Burundi was hitting back at France over a statement criticising a referendum earlier this month which reformed the constitution, allowing President Pierre Nkurunziza to seek another two terms in office and remain in power until 2034.
The diplomat said a similar project in Ruyigi province financed by Belgium, had not met any problems.
Burundi was plunged into crisis in 2015 when Nkurunziza sought a third term in office, sparking protests and violence that left least 1,200 dead and 400,000 displaced.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.) Click for more trending news