The country's ministry of planning and foreign aid said Oxfam GB had made a "serious error" by failing to inform Haitian authorities of the actions by their staff at the time they occurred.
"These reprehensible acts, alleged crimes, acknowledged by the perpetrators as well as the NGO, are a serious violation of the dignity of the Haitian people," a government statement said.
It said the charity was suspended for two months pending an internal Haitian investigation into the matter.
Aviol Fleurant, the minister for planning and foreign aid, said that if Haiti's own investigations found "a link between the crimes committed and the money that Oxfam received to help the Haitian people I will see to it that the foreign ministry declares, in the name of the government, Oxfam-GB persona non-grata and will have to leave the country."
"Haiti is no longer an NGO republic," he said after meeting with Oxfam's director for Latin America and the Caribbean, Simon Ticehurst. "We are not joking: the dignity of the Haitian people has no price."
An investigation conducted by Oxfam a year after the earthquake had found that seven staff were accused of using prostitutes at an Oxfam-funded residence.
The 2011 report, disclosed on Monday, also found that three staff members had physically threatened a witness.
One official has acknowledged paying for prostitutes and the others were accused of harassment and intimidation.
A young Haitian woman told The Times newspaper she had had relations with the former Oxfam country director in Haiti, Roland Van Hauwermeiren, when she was 16 and he was 61.
Van Hauwermeiren denied to Belgian media last week, however, that he had organized orgies with young prostitutes in Haiti, and said he had "intimate relations" with "an honorable, mature woman" whom he did not pay.
Oxfam formally apologized to the Haitian government on Monday for its handling of the scandal and for failing to report it promptly to local authorities.
"It was not for Oxfam to decide whether a crime had been committed... that was the wrong decision," Oxfam GB's chief executive Mark Goldring said Tuesday at a hearing before a British parliamentary committee.
Fleurant, the planning minister, said he had not ruled out demanding the extradition of Van Hauwermeiren and others to face charges in Haiti.
"There could be an extradition: we could call for them to be brought back to Haiti," he said, adding that if that did not succeed "we could bring a civil action, or help the victims do so."
The strong line taken by the government was met with approval by Haitian human rights groups.
"It's a good decision and it also sends the right message to other NGO's operating in the mess of Haiti," said Haitian lawyer Mario Joseph, while warning that such talk "could be much ado about nothing if all the means to carry out an inquiry are not deployed to bring these people to justice."
The Haiti scandal has opened the charity to greater scrutiny, leading to 26 new cases of sexual misconduct, most involving its international operations.
(This story has not been edited by NDTV staff and is auto-generated from a syndicated feed.)
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.